-- Logs begin at Tue 2022-07-12 13:48:43 UTC, end at Tue 2022-07-12 14:53:03 UTC. -- Jul 12 14:18:12.608932 dv-cinder-836996-39-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 14:18:12.620629 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 14:18:12 2022] *** Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: nodename: dv-cinder-836996-39-iscsi Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: machine: x86_64 Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: clock source: unix Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: pcre jit disabled Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: detected number of CPU cores: 4 Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: current working directory: / Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: detected binary path: /usr/bin/uwsgi-core Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: your processes number limit is 47680 Jul 12 14:18:12.622690 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: your memory page size is 4096 bytes Jul 12 14:18:12.623409 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: detected max file descriptor number: 2048 Jul 12 14:18:12.623409 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: lock engine: pthread robust mutexes Jul 12 14:18:12.623409 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: thunder lock: enabled Jul 12 14:18:12.623409 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 14:18:12.623409 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 14:18:12.667061 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: Python main interpreter initialized at 0x55f8377679c0 Jul 12 14:18:12.667061 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: python threads support enabled Jul 12 14:18:12.667061 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: your server socket listen backlog is limited to 100 connections Jul 12 14:18:12.667061 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: your mercy for graceful operations on workers is 90 seconds Jul 12 14:18:12.667372 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 14:18:12.667443 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: *** Operational MODE: preforking *** Jul 12 14:18:12.667685 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: *** uWSGI is running in multiple interpreter mode *** Jul 12 14:18:12.667685 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: spawned uWSGI master process (pid: 80383) Jul 12 14:18:12.667873 dv-cinder-836996-39-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 14:18:12.673241 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 1 (pid: 80393, cores: 1) Jul 12 14:18:12.673241 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 2 (pid: 80394, cores: 1) Jul 12 14:18:12.673241 dv-cinder-836996-39-iscsi devstack@keystone.service[80383]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 14:18:18.097309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:18:18.097309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 12 14:18:18.098095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:18:18.098095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jul 12 14:18:18.098509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:18:18.098509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes Jul 12 14:18:18.098615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 14:18:18.098615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jul 12 14:18:18.146195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 14:18:18.146511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 14:18:18.146608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 14:18:18.146692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] command line args: [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 14:18:18.146874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 14:18:18.146960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 14:18:18.147073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] command line args: [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 14:18:18.147158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ================================================================================ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 14:18:18.147353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 14:18:18.147431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.147534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ================================================================================ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 14:18:18.147612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_dir = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.147769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.147857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.147954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_source = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_dir = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] debug = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_source = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] debug = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.148918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.149784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_config_append = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_config_append = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_dir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.150990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_dir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.151876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.152993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.153978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] publish_errors = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.154955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] publish_errors = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.155886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] standard_threads = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] standard_threads = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_journal = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.156977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_json = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_journal = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_stderr = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_json = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_syslog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_stderr = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_syslog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.157775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.157853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 14:18:18.158013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.158976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.159928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.160954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.161865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.162962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.163885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.164967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.165045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.165193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.165268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.165446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.165575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.166979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.167980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.168904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.169880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.170975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.171889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.172891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.173976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.174918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175232 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.175967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.176906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.177862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.178947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.179946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.180873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.181861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.182965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.external = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.183926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.external = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.password = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.token = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.184981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.password = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.token = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.185914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.186984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.187925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.188986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.189999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.190964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.191884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.192976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.193960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.194968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.195898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.196856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.197852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.198948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.199968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.200929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.201874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.202964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.203980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.204906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.205864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.206972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.207981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.208915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.209899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.210972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.211998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.212146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.212226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.212889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.212976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.213952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.214876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.215990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.216996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.217953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.218940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.219899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.220925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.221940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.222984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.223911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.224851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.225958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.226988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.227924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.228911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.229908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.230875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.231951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.232954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.233914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.234006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 14:18:18.234084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.234301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.234537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 14:18:18.234708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 14:18:18.265249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.265502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.265722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.265807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.266028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.266260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.274061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.274780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.302791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.303014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.303144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.303283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.445732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.447240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.495144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.495368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.495459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.495668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.559955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.560954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.561084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.561168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.561301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.561544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.585796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.585949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.586935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.587063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.587146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.587271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.598312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.598439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.611391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.611617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.611753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.611873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.611950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.612874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 14:18:18.629869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.629991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.630072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:18:18.630147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:18:18.630464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.630616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.630702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:18:18.630815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 14:18:18.630901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.631848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.643915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.644052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.652367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.652763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.655648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.655921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.656177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.656933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.657829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.658806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.685678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.686370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.687233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.687508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.687985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.688262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.692762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.693039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.693315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.693576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.693854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.694253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.701259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.701558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.702117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.702396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.706035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.706317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.706589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.706862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.706944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.707131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.707924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.708008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.708219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.708486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.708751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.726697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.726968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.727831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.728627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.732175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.733364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.735796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.736084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.736998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.737288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.741219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.742470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.744901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.746182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.748558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.748847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.749130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.749444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.749726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.749852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.750000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.750150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.750442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.750732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.751008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.751284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.769086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.769389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.769667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.769924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.770496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.770777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.771042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.771302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.778360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.779776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.780046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.780316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.781494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.781766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.783504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.783779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.784055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.784330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.784601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.784943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.785221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.785523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.785803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.786079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.797963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.798238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.798512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.798797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.799073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.799356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.799475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.799631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.799754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.800032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.800565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.800841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.802427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.802729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.819569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.821237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.822905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.824587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.826237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.827824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.827909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.829617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.829826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.829908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.830162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.831301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.831579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.831865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.838112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.839722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.839808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.841458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.842994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.843277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.843555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.843824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.844590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.844875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.845263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.845571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.853438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.853722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.853999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.854772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.855054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.855328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.861868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.862141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.862414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.862682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.862946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.863219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.863302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 14:18:18.863484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.863569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.863760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.863845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.864896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.865743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.866010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.866271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.866534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.866806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 14:18:18.904411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f8377679c0 pid: 80394 (default app) Jul 12 14:18:18.905495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f8377679c0 pid: 80393 (default app) Jul 12 14:18:18.907793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34a08e-75c2-4529-84e6-cab0fde84224 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:18.908084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34a08e-75c2-4529-84e6-cab0fde84224 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:18.908362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34a08e-75c2-4529-84e6-cab0fde84224 None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:18.912516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Tue Jul 12 14:18:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:26.487116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:26.487669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:26.487669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:26.491944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:26.497599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:26.498021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:26.498158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:26.566547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: INFO dbcounter [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] Registered counter for database keystone Jul 12 14:18:26.588016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG oslo_db.sqlalchemy.engines [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 14:18:26.594280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writer thread running {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 14:18:26.769883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 14:18:26.770227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 14:18:26.824483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:26.826132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:26.945932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 14:18:27.140354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:27.219721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: INFO dbcounter [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] Registered counter for database keystone Jul 12 14:18:27.239394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG oslo_db.sqlalchemy.engines [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80393) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 14:18:27.245553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writer thread running {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 14:18:27.363851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660faa37-af72-4b53-81ed-c0f21ff042ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:27.368116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:27.369320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:27.369629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:27.369909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:27.377932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:27.430171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 14:18:27.430523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34c3a-f1e3-4bb4-9681-6524416d08c1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:27.441251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Jul 12 14:18:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:18:27.447193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:27.462306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c2fc7-5b7e-4c17-934e-2da2ab2294ff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:27.466799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:27.467924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:27.468215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:27.468493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] PATH_INFO: `/v3/projects/63dcdffb6815414f8835106f58a82105` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:27.472587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] RBAC: Authorizing `identity:get_project(project_id=63dcdffb6815414f8835106f58a82105)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:27.524327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 14:18:27.525467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbba77-4081-41a0-b57a-d80c5df0b295 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:27.530003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Tue Jul 12 14:18:27 2022] GET /identity/v3/projects/63dcdffb6815414f8835106f58a82105 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:29.531883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161920-ac66-47b1-b039-9c454f24e7ea None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:29.532402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161920-ac66-47b1-b039-9c454f24e7ea None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:29.532496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161920-ac66-47b1-b039-9c454f24e7ea None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:29.536563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:29.542606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:29.543026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:29.543167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:29.562441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:29.563942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:29.572338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 14:18:29.594196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161920-ac66-47b1-b039-9c454f24e7ea None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:29.606260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161920-ac66-47b1-b039-9c454f24e7ea None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:29.610512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:29.611144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:29.611435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:29.611719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:29.620344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:29.622200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a993d1-2eea-4d67-978f-860954617158 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:29.631029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Jul 12 14:18:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:18:29.636766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:29.645792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d89eec-ff11-4e9d-bdd2-11c3343ea640 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:29.650156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:29.651021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:29.651304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:29.651582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:29.652967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:29.654914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa6260-f47e-44ca-8fe1-98f6bb992add admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:29.656476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Tue Jul 12 14:18:29 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:31.727326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea321-6731-4603-9672-e38e28de2b17 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:31.727875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea321-6731-4603-9672-e38e28de2b17 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:31.727875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea321-6731-4603-9672-e38e28de2b17 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:31.729276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:18:31.734396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:31.734798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:31.735025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:31.744197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204015-84ed-4337-9e58-1b92c9086aa1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:31.744484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204015-84ed-4337-9e58-1b92c9086aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:31.744765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204015-84ed-4337-9e58-1b92c9086aa1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:31.746106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:18:31.751019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:31.751300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:31.751576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:31.752807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:31.754027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:31.791833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80393) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 14:18:31.792247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80393) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 14:18:31.837796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:31.838331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:31.839931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:31.863581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:31.887718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:31.940627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:31.963295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:31.966741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83d450-5f63-442a-acde-1bb4df8c0738 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:31.999290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.000020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.000306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.000588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.012100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:32.023224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:32.024277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:32.025846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:32.032471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.062478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c9e8e-4335-4cef-b819-333a1d997bb1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.064770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.100215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.101111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.101420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.101704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.114373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.133559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c6cc0-f7c2-4e37-aa93-3f299563b2ab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.168724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.169423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.169714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.169988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.170538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.170744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:32.171052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.172073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.172434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.184989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:32.190720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.269284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:18:32.275230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.277955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.289008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5ef0-54d8-49a9-8b59-81549fea155c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.323223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.324102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.324389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.324668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.325918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.327383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.329008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:32.334397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.348779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.374169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3504353-a095-453b-b0ce-cdf0dc61443d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.407421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.408211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.408491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.408765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.427625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.429092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.436240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446c2c9-960e-41d8-ba24-afb3f61afb7e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:18:32.451285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:32.452266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:32.453133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:18:32.453133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:18:32.453133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:18:32.453133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:32.453133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:32.458341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.472375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.473170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.473489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.473766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.474934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.476375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.478000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:32.483964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:32.513923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.558013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1808d50-5f1a-4790-a2d1-562cd5c9358d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.565299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:32.584782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a66a-482b-4838-a7df-2c9a99141e25 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:32.591834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.592505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.592793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.593073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.593834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:32.594148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.595547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c24c-02d5-4d3b-8785-da3fa06b0b86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.622827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:32.623059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:32.623059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:32.623235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:32.623464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:32.624323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.domain.id=default, user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:32.624641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:32.626165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e1d4-d489-4074-a7ad-3ab3127ab715 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:32.634531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Jul 12 14:18:32 2022] GET /identity/v3/role_assignments?user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7&scope.domain.id=default&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:34.915773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142bb3d-1119-4264-bc96-d25d859dd768 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:34.916797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142bb3d-1119-4264-bc96-d25d859dd768 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:34.917313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142bb3d-1119-4264-bc96-d25d859dd768 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:34.918971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:34.924507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557301-59b9-47eb-8f27-c908f3021fb7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:34.925351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557301-59b9-47eb-8f27-c908f3021fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:34.926073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557301-59b9-47eb-8f27-c908f3021fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:34.946302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92557301-59b9-47eb-8f27-c908f3021fb7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:34.947788 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92557301-59b9-47eb-8f27-c908f3021fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:35.042631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:35.081178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142bb3d-1119-4264-bc96-d25d859dd768 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:35.101194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaf974-8583-490e-9409-ffebac227604 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.101843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaf974-8583-490e-9409-ffebac227604 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.102314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaf974-8583-490e-9409-ffebac227604 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.104608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:35.109799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.110305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.110796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.138342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:35.139921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:35.366084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142bb3d-1119-4264-bc96-d25d859dd768 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:35.378772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:35.394228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:35.430012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:35.432689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:35.433762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.434254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.434788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.438376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:35.460428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:35.462675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4f408b88-80c7-4fe0-af35-f345fc7ba399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:18:35.485713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:35.486599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:35.487351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:18:35.487351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:18:35.487351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:18:35.487351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:35.489272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 12/26] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:35.490811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d109-4078-43f5-ba90-196963e30ea2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.491309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d109-4078-43f5-ba90-196963e30ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.491789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777d109-4078-43f5-ba90-196963e30ea2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.493800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:35.502577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777d109-4078-43f5-ba90-196963e30ea2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:35.666869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777d109-4078-43f5-ba90-196963e30ea2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:35.738621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:35.740220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.741278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.741971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.743583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:35.744335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:35.746565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9b02b-5f97-4de1-9eb5-0425231b328a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:35.762632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596871c-39f3-45e2-9146-535a76ca3ea8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:35.770805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 14/28] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:35.772706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.776719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.777231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.814420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:35.815642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:35.828887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:35.831191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:35.831752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:35.832233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:35.840947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:35.855403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:35.863238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:35.864080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:35.864873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 15/29] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:35.865608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:35.933048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 15/30] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:35.934026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:35.998266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.010315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba002-1411-4811-9437-931bddc80ccf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.048290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fe31f-8a42-4af5-817c-774aa964988c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.049236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.049966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.050256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.050923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.051605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.053192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.087988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.089068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.089657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.090102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.091075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:36.091545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.093161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f80b1-87ff-45ff-959e-b03d8f47eb08 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.112096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 16/31] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:36.113852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0d91b-ab31-4398-9ca9-8eb4f1d1b7ca None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.114358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0d91b-ab31-4398-9ca9-8eb4f1d1b7ca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.114805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0d91b-ab31-4398-9ca9-8eb4f1d1b7ca None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.116395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 17/32] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:36.117259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0d91b-ab31-4398-9ca9-8eb4f1d1b7ca None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.142872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.228158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 16/33] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:18:35 2022] POST /identity/v3/projects => generated 301 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:36.229367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.234060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0d91b-ab31-4398-9ca9-8eb4f1d1b7ca None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.336481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.338632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.339382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.339927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.340884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.342578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d04d57-ed7f-447a-bc5a-78bebec36fa2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.368847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874138e1-613b-419e-910a-6aea9df0d2d0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.389712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.414406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.415414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.415706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.415987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.417545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.419323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaf807-b973-4fcf-8477-8545b2fce887 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.421132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 17/34] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:36 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:36.422576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78e9c-713e-4505-9914-368eeb008956 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.422876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78e9c-713e-4505-9914-368eeb008956 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.423219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f78e9c-713e-4505-9914-368eeb008956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.444428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66f78e9c-713e-4505-9914-368eeb008956 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:36.446813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66f78e9c-713e-4505-9914-368eeb008956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:36.500540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 18/35] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/roles => generated 219 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:36.502728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f532b4-8e9d-4db8-8c80-4cb96b532161 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.506565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f532b4-8e9d-4db8-8c80-4cb96b532161 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.509865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f532b4-8e9d-4db8-8c80-4cb96b532161 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.511359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 19/36] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:36.512121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f532b4-8e9d-4db8-8c80-4cb96b532161 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.524238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 18/37] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:36.526136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.526819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.527348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.548908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:36.550873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:36.616851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.642892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 19/38] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:36.643755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.654374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f532b4-8e9d-4db8-8c80-4cb96b532161 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.698764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.708163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.709127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.709302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.709596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.711236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.712871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.714746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 20/39] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:36.715364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.766719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13494b37-b27c-4bc8-bd8d-84d86da0851c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.806635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.807318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.807601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.807886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.808635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.810093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344e8ca-c93c-4458-8217-8e3f66d0e7a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.825619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e143ab-4ba6-427a-9f00-c995d1e56f3e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:36.828721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 20/40] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/roles => generated 225 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:36.830188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5340fb-19c8-4a21-ae99-2a0037d4dc3b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.830477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5340fb-19c8-4a21-ae99-2a0037d4dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.830756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5340fb-19c8-4a21-ae99-2a0037d4dc3b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.833383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 21/41] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:36.833383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5340fb-19c8-4a21-ae99-2a0037d4dc3b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:36.869748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:36.870579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.870874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.871159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.872479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:36.874054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b06e8-bfe5-4ab7-a43d-399ad18a06e0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:36.876058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 21/42] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:36.877119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:36.877442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:36.877755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:36.896671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:36.898345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:36.912189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:36.981542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 22/43] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:36.990772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:37.072107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5340fb-19c8-4a21-ae99-2a0037d4dc3b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:37.074505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:37.097245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e2a6d-3922-4ef7-ae11-75d936021c7d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:37.107988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:37.108870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:37.109164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:37.109481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] PATH_INFO: `/v3/domains/default/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:37.111650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:37.113166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:37.127228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 22/44] 10.222.0.25 () {58 vars in 1361 bytes} [Tue Jul 12 14:18:36 2022] PUT /identity/v3/domains/default/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:37.128012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:37.144223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:37.145025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:37.145320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:37.145642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:37.146382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:37.147904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31054a-6667-4b01-9aa9-f080da127144 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:37.174981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:37.187362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 23/45] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:18:36 2022] POST /identity/v3/projects => generated 312 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:37.260123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003bb7f-948b-4efb-a308-21013552f126 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:37.304734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:37.305585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:37.306479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:37.306834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:37.307609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:37.309048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca96f2-9628-45d7-9d2f-65e6a4e27f67 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:37.325518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 14:18:37 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:39.175184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa41-c15b-48e8-a206-22c7fb5585fe None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.175990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa41-c15b-48e8-a206-22c7fb5585fe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.176074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa41-c15b-48e8-a206-22c7fb5585fe None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.177443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 24/47] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:39.183198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.183627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.183889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.200831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:39.202081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:39.219895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:39.241168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f53bd-7355-4988-97dc-5a6586120698 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.241495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f53bd-7355-4988-97dc-5a6586120698 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.241788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f53bd-7355-4988-97dc-5a6586120698 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.243164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 25/48] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:18:39.248072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.248360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.248644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.265432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:39.266622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:39.270303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 24/49] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:39.294207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.401082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7847e-23f1-4c5f-9641-267701c0a826 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.437366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.438108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.438400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.438683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.439377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 26/50] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:39.449920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:39.457607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:39.458497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:39.459241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 25/51] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:39.462967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.464968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.480392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:39.563574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2093e56-5955-4f09-9fca-57314966bb30 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.568554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842330b4-f92a-420b-892e-7f7073708ead None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.589357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:39.597615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.598408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.598698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.598982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.600158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:39.601646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:39.603764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 27/52] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:39.604199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.604897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.605159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.605478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.606249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:39.606566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:39.607968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:39.609487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.616842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:39.622610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.703297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:39.708185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f21ffa-e1b7-4df4-bf3f-4062a2d5379a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.728689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aec5d0-d3c6-487d-b351-982e918b5227 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.742787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.743975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.744261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.744569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.745301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:39.746787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:39.761814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.762766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.763051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.763327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.764610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:39.766582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:39.768216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 27/54] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:39.771776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 28/55] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Jul 12 14:18:39 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:18:39.778458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.885152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa618e5-1871-4697-a2a1-a52d58242e1d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:39.920723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:39.921693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:39.921836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:39.922113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:39.923378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:39.924850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d546bb-7f90-42ee-8f50-b39576043e00 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:39.926828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 28/56] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:39.932660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:39.985942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:40.041827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ccc51-7574-4e53-954d-23b3ede615f9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:40.075781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:40.076437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:40.076721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:40.076997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:40.077855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.domain.id=default, user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:40.078168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:40.079609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0c42c-c839-44a9-b3a5-a6fec5648fdd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:40.088694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 29/57] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Jul 12 14:18:39 2022] GET /identity/v3/role_assignments?user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7&scope.domain.id=default&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:41.852312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25e1b-b2c9-425d-881f-35972ab8a2d1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:41.852818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25e1b-b2c9-425d-881f-35972ab8a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:41.852919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25e1b-b2c9-425d-881f-35972ab8a2d1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:41.854795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 29/58] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:18:41.859875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:41.860289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:41.860436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:41.879365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:41.880563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:41.946594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 30/59] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:41.971666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f25e1b-b2c9-425d-881f-35972ab8a2d1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:41.987322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:42.077542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f25e1b-b2c9-425d-881f-35972ab8a2d1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:42.110506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:42.111231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:42.111518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:42.111797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:42.112981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:42.114481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069add5-d10d-4a63-b77d-bf1ec6b2338d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:42.116110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 30/60] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:42.122297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:42.164473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:42.222059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105827-9462-4d40-b3eb-01b0c64fd9b0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:42.255159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:42.255828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:42.256115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:42.256395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:42.257111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:42.259131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf3d5d-5e84-435e-bfce-ad4cd7254821 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:42.285284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 31/61] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:18:42 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:45.584725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03e6-16fb-442c-a906-e5684ed85d09 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.585589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03e6-16fb-442c-a906-e5684ed85d09 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.586043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03e6-16fb-442c-a906-e5684ed85d09 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.588441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 31/62] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:45.593415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.594177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.594608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.619303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:45.620796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:45.715797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 32/63] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:45.749479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2855-2cef-4f6b-a710-2f7c56404301 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.749991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2855-2cef-4f6b-a710-2f7c56404301 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.750443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e2855-2cef-4f6b-a710-2f7c56404301 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.752026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 32/64] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:45.753249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:45.762828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f85c89-30af-4112-b1c9-03f2e0aa386c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.763411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f85c89-30af-4112-b1c9-03f2e0aa386c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.763932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f85c89-30af-4112-b1c9-03f2e0aa386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.785027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:45.787440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-47f85c89-30af-4112-b1c9-03f2e0aa386c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:45.788861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47f85c89-30af-4112-b1c9-03f2e0aa386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:45.863326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 33/65] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:45.864748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fe2e-32a5-4504-bf4d-9ba8e4b60148 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.865050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fe2e-32a5-4504-bf4d-9ba8e4b60148 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.865346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fe2e-32a5-4504-bf4d-9ba8e4b60148 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.866770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 34/66] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:45.871405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635750dd-1702-4488-ac75-46acfcf3e3f2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.871898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635750dd-1702-4488-ac75-46acfcf3e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.872257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635750dd-1702-4488-ac75-46acfcf3e3f2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.874821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 35/67] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:45.874821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.877370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.877370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.877568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67232b48-ac57-44e9-8634-c39e08f6e7f6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:45.895294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:45.896494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:45.899537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:45.919123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:45.919922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.920212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.920504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.932339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1986cacd-3ad2-41e1-9b2b-65b294e2e743 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:45.941262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:45.942212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:45.942957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 33/68] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:45.945276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e001-241a-4b88-8a8a-117ea381dd21 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:45.945276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e001-241a-4b88-8a8a-117ea381dd21 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:45.945276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e001-241a-4b88-8a8a-117ea381dd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:45.962583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f8a1e001-241a-4b88-8a8a-117ea381dd21 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:45.963800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f8a1e001-241a-4b88-8a8a-117ea381dd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:45.967265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 36/69] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:45.967904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.032257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 34/70] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:46.033627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e079c-1f25-4660-aa85-28603361a9e6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.033922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e079c-1f25-4660-aa85-28603361a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.034197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e079c-1f25-4660-aa85-28603361a9e6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.035483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 35/71] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:46.036183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e079c-1f25-4660-aa85-28603361a9e6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.055083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.073918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565bd5a-06bf-41ff-bc10-2dce938db182 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.108721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.109543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.110496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.110496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.118729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.122392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:46.130181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:46.131182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:46.131932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 37/72] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:46.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.145940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e079c-1f25-4660-aa85-28603361a9e6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.181196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.181888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.182172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.182448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.183189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:46.183493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.184875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7ce3f-97b5-4ee2-8a87-9ef9869d1c26 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.193038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.194344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.194691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.194974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.212560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:46.213780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:46.232291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80147ccf-eef3-4484-bf6f-1da508edad70 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.264822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.265653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.265940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.266218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.276855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:46.279197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 37/74] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:46.279799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:46.284541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:46.285402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:46.286162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 38/75] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:46.286794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.291670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.355637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.378533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a43c0-e975-4113-8f1d-1f88e6bffb24 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.397194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14977-723a-413a-82ed-b184ed581c80 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.413669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.414759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.415192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.415606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.426934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:46.431851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.432521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.432810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.433091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.433872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:46.434189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:46.435466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:46.436307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:46.437038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:46.437654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.437717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:46.437717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.448988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.449897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.506203 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.537569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d010e4-cc1b-40de-bc7b-13df335394a9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.546787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d39d4-2f84-4402-a0e8-cb2ded1f62bb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.569139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.570778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.572225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.572511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.572796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.581864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.582538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.582827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.583106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.583866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:46.584177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.585586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.588147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.589623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.591323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 39/78] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.592097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.594451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 40/79] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.595399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.695471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5d88-e693-4764-a400-63a361687bcc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.699095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007829-f648-4be3-bcad-1660ed2f7d37 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.722192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.730262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.731059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.731347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.731632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.732804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.732968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.733670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.733958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.734235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.734320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.734975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:46.735283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.735900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 40/80] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.736739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.741030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.749114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 41/81] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.749977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.795430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.837281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8454f-1816-4e41-97eb-fc4f4b619a34 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.847681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fd64f-b85a-47c9-9364-90c245737eb0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:46.872109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.872979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.873266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.873595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.874799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.876259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.877929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 41/82] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:46.877929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:46.885321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:46.886006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:46.886299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:46.886581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:46.941696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:46.954887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:46.956394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:46.979323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65c026-e992-43bd-acd4-a473fbc50b05 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.002314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.015696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.016584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.016876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.017168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.018436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.019881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.021563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 42/83] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:47.022400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.060137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 42/84] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:47.061030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.123936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b4c2-1a36-451d-8724-8bd5d6353193 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.152874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.157554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5a738-a263-4c16-8c85-dfe37a95a336 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.158443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.159108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.159392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.159674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.160387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.161867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.181664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 43/85] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:18:47 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:47.182475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.195932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.197762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.198200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.198817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.200467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.202335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.204360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 43/86] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:47.205199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.226543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.303591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301887ab-c672-4ada-9656-2329ab806582 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.319759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289e856-49bb-49c3-a49a-cff85907e97f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.344397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.345181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.345513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.346453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.360269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.360902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.361190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.361542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.413732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.431875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.433391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.463406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.465028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.485134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.550955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 44/87] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:47.551915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.600238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 44/88] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:47.601726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.673798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.689669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc65d-a7de-4cc3-ad75-55aef07847f2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.727588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f9db-548a-4b58-a106-81b80806b1fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.733104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.734517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.734939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.735352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.736919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.738757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.741032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 45/89] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:47.741971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.744042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.765723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.766557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.766961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.767320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.840346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.842173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.855353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37f9c-c092-4c84-a971-262a4e3b56b2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:47.891143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:47.892248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:47.892672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:47.893112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:47.894891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:47.896482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:47.899632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 46/90] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:47.900795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.936347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:47.955186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 45/91] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:47.960526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:47.989715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:48.021471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a510ca-016d-4e9a-92f6-d07b4fefe157 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.059978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.061111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.061565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.061983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] PATH_INFO: `/v3/projects/6f8bf0894eb14221b351a195eb205a69` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.063324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] RBAC: Authorizing `identity:get_project(project_id=6f8bf0894eb14221b351a195eb205a69)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.064942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.066335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb061c0-23a0-4771-9c0c-6285a0b9cc16 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.067081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 47/92] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/projects/6f8bf0894eb14221b351a195eb205a69 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:48.068026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:48.103389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.104205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.104559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.104946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.105937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:48.106818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.107759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.114424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 46/93] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:47 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=37d514b6250f41f1a7a945e02510a6df => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:48.115356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:48.172531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:48.184163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b72d1-69f1-499b-9a64-e056662ed182 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.223408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.224504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.224985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.225511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.225724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:48.226766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:48.227925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.230486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.238396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 48/94] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:48 2022] GET /identity/v3/role_assignments?user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:48.239567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:48.262900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5442b9-b401-4931-94c3-02dc756f7854 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.330430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.330791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.331002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.331279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.333008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.334807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.337602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 47/95] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:48 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:48.343726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:48.455568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441353a-2d50-4273-9c03-9e916f884562 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.478431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c602f-50e4-4578-b732-365441beb4c5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:48.519715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.520695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.521740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.522207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.523541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:48.524640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.526697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c390-31fc-4ce2-8907-272e61b7c45c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.534528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 48/96] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:48 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:48.544869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:48.545928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:48.546420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:48.546890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:48.548006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=6f8bf0894eb14221b351a195eb205a69, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:48.548574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:48.550325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c73c40-fd8e-4610-a738-30a74e0235d9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:48.565528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 49/97] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:48 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=6f8bf0894eb14221b351a195eb205a69&role.id=37d514b6250f41f1a7a945e02510a6df => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:49.858922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64690-0cf2-4521-85af-6ada31de1d72 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:49.859856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64690-0cf2-4521-85af-6ada31de1d72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:49.860328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d64690-0cf2-4521-85af-6ada31de1d72 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:49.862068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 49/98] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:49.867476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:49.868361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:49.868862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:49.891595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:49.893118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:49.913449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:49.945721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bc39-f77e-49fb-8fe0-c3feca88ea93 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:49.946406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bc39-f77e-49fb-8fe0-c3feca88ea93 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:49.946910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bc39-f77e-49fb-8fe0-c3feca88ea93 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:49.948484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 50/99] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:49.953502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:49.954025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:49.954506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:49.985109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:49.987846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:49.997765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:50.021464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 50/100] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:50.063096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:50.095940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 51/101] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:50.130570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:50.269622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31039e46-f520-4062-b44c-7d5d5f989b5d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:50.350062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:50.350877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:50.351169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:50.351450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:50.353189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:50.354675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:50.356626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 51/102] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:50.362463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:50.369184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e7336-e6ba-4f03-990b-3cbd62e54d4b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:50.402885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:50.470067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:50.470889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:50.471171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:50.471447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:50.496619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:50.499065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:50.507430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:50.508912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:18:50.510437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:50.515066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 52/103] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:50.520706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:50.651616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53695-4e8a-4832-81d1-b2ee769f0a90 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:50.710483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8c60-21ff-4f49-a1e7-a09bd01dcaee None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:50.751407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:50.752087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:50.752380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:50.752664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:50.753413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:50.754874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:50.775864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:50.776646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:50.776944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:50.777226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:50.778096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:50.778409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:50.779787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:50.794313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 53/104] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:50.799826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:50.810023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 52/105] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:18:50 2022] POST /identity/v3/projects => generated 302 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:18:50.902994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:50.958028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a41d-94a4-4496-8185-181839fd2a0c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.002506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.003331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.003617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.003900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] PATH_INFO: `/v3/users/0da7e48d096c4a8d8290d78e5d0a640b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.019660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] RBAC: Authorizing `identity:get_user(user_id=0da7e48d096c4a8d8290d78e5d0a640b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:51.021154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddaaf45-7868-4688-9462-2f7146b86811 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:51.023282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 54/106] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:50 2022] GET /identity/v3/users/0da7e48d096c4a8d8290d78e5d0a640b => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:51.028751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.036034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0f394-a8ea-404b-b36a-533613ccc776 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.036034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0f394-a8ea-404b-b36a-533613ccc776 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.036034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0f394-a8ea-404b-b36a-533613ccc776 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.036705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 55/107] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:51.042666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.043237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.043766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.072340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:51.074012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:51.114174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.176076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ef582-602f-4305-bcb9-40d56a796cb4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.211617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 56/108] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:51.229589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f278-4899-483b-b800-cd651358a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.230252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f278-4899-483b-b800-cd651358a76a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.230550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f278-4899-483b-b800-cd651358a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.230835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f278-4899-483b-b800-cd651358a76a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.253585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.347859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.358833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467f278-4899-483b-b800-cd651358a76a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:51.360338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467f278-4899-483b-b800-cd651358a76a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:51.435125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25c7f8-b688-4a28-b8e9-336c5d374f74 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.472923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.473772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.474062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.474342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.487483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0e9684dc-ddb6-42a8-b923-816183ae2100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:51.496199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:51.497111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:51.498298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 57/109] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:51.501526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fb13-8434-4dd0-83b6-3bc1c17c4637 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.501526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fb13-8434-4dd0-83b6-3bc1c17c4637 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.501526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355fb13-8434-4dd0-83b6-3bc1c17c4637 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.501526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 58/110] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:51.509317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.509980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.510495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.521475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.535579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:51.536793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:51.572644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 53/111] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:51.573298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f278-4899-483b-b800-cd651358a76a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.609963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 59/112] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:51.610936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.660394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.691666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f278-4899-483b-b800-cd651358a76a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.727258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a78bc-da4a-46e4-8fbc-467d29e43453 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.731589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.732264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.732554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.732839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.733648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:51.733960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:51.735350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50511e78-aebd-4f35-b3f3-ddab96d4eb6b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:51.744228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 54/113] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:51.745484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb30608-a7da-45b2-a8f2-a621b4a8758c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.745778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb30608-a7da-45b2-a8f2-a621b4a8758c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.746060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb30608-a7da-45b2-a8f2-a621b4a8758c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.751221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 55/114] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:51.751221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb30608-a7da-45b2-a8f2-a621b4a8758c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.776626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.777910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.778202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.778748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.780112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:51.780576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.782315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895486ad-c9df-45be-9ca0-85e357c0d2fe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:51.784148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 60/115] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:51.785592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04628fdf-60e4-4d79-9ae3-0c51ac2d2557 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.785884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04628fdf-60e4-4d79-9ae3-0c51ac2d2557 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.786163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04628fdf-60e4-4d79-9ae3-0c51ac2d2557 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.788263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 61/116] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:51.788989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04628fdf-60e4-4d79-9ae3-0c51ac2d2557 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:51.862079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb30608-a7da-45b2-a8f2-a621b4a8758c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.899966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.900759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.901052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.901353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.901874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04628fdf-60e4-4d79-9ae3-0c51ac2d2557 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:51.911875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:51.912443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-00f21307-520b-4e7c-9ae1-aa45c282c93b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:51.920538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:51.921454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:51.922193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 56/117] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:51.923539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668ea4-c029-4981-8e65-2ba166d82354 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.923539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668ea4-c029-4981-8e65-2ba166d82354 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.923539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668ea4-c029-4981-8e65-2ba166d82354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.940702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-34668ea4-c029-4981-8e65-2ba166d82354 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:51.941925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34668ea4-c029-4981-8e65-2ba166d82354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:51.947218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:51.948097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:51.948385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:51.948661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:51.949876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:51.951324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:51.952959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 62/118] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:51.953753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.020277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 57/119] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:52.021684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.021979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.022263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.048802 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:52.049962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:52.069069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.069247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e224ae2-04ff-4d16-a8f5-9c8133d719ec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.105982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.106650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.106946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.107230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.108080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=0da7e48d096c4a8d8290d78e5d0a640b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:52.108398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.109893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.117448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 63/120] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:51 2022] GET /identity/v3/role_assignments?user.id=0da7e48d096c4a8d8290d78e5d0a640b&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.117448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.137774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 58/121] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:52.139168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.167004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.251497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a0b5-2904-4ca7-ab46-51c0b1a5c846 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.276018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d89665-8ae0-4059-a9a8-a17f85d80015 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.292794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.293696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.294118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.294502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.295439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:52.295759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.297958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.309077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 64/122] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.310235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.328551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.330075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.330829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.331348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.333225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.335450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.338638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 59/123] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.344378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.370973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.431261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec33283-eaa0-4114-8cb0-b7f0bfd0f3a4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.433152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.471044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.471866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.472154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.472433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.481350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d165a8-5d82-4b2a-9428-5f9f3c965c93 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.485623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:52.494878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:52.495822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:52.496601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 65/124] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:52.497291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.533311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.535136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.535690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.536419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.551525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:52.563330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:52.564213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:52.564979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:52.566694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 60/125] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:52.567484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.623345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7932e-3ada-4b58-a376-b8027f025747 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.658653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.659547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.659832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.660167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.661472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.663068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.670036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 66/126] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.671418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.677707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.687968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4356f0-5eff-4f1a-9c17-7ef801a9344f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.707588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.734888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.736295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.736995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.737452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.740465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.742841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.761418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 61/127] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:18:52 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:52.762872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.789313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212b6d1-4e8b-48f5-a5f1-045f48bc1d01 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.832928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.833602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.833903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.834219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.835057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:52.835374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.837043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.847069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 67/128] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.848193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.902844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167777e-9d74-45ea-af94-37c3bcfceede None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:52.941918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:52.942395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:52.942692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:52.942978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:52.943748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:52.944062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:52.945634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:52.960270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 62/129] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:52.961197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:52.967696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.968215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:52.985882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0ff13-c082-4128-a601-1db838f5956e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.050485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.051692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.051981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.052263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.053610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.055272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.061493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 68/130] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:53.062481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.111699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27311-1c69-461b-8f03-908612ade8be None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.151685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.153009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.153536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.154062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.155471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.157113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.159171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 63/131] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:52 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:53.160161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.181924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0256b1-9ffb-4cf4-8ceb-94f99c40c990 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.225059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.226089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.226384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.226694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.228131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.229622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.231423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 69/132] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:53 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:53.236673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:53.237628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.276696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05908c-0595-452e-b53a-87f7729065e5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.292912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:53.343502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.344593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.344974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.345363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.348701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, user_id=9a6ac7756ddb4d96967796f1a0d107a7, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.351572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.374240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 64/133] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:18:53 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:53.374958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.503891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e13604-859c-4251-ad15-13a7551de76a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.574863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.575920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.576306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.576773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.578784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.581139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.584836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 70/134] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:53 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:53.588887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.591375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779b6b-6f3d-4c64-b480-e97868249c3a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.630880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:53.641960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.644054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.644589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.645080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] PATH_INFO: `/v3/projects/6f8bf0894eb14221b351a195eb205a69` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.646542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] RBAC: Authorizing `identity:get_project(project_id=6f8bf0894eb14221b351a195eb205a69)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.648744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.651660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 65/135] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:53 2022] GET /identity/v3/projects/6f8bf0894eb14221b351a195eb205a69 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:53.658254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:53.730375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:53.751294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15013912-57c8-40fb-8285-64bf90f15cf4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.783856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4e27c-66f3-47b5-92d1-592d330396b0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:53.823874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.825422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.825912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.826757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] PATH_INFO: `/v3/projects/6f8bf0894eb14221b351a195eb205a69/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.829102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] RBAC: Authorizing `identity:create_grant(project_id=6f8bf0894eb14221b351a195eb205a69, user_id=9a6ac7756ddb4d96967796f1a0d107a7, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.831007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c6041-1703-4f42-a6c3-6f9ed519247b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.837158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:53.838235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.838536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.838808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.841158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, user_id=9a6ac7756ddb4d96967796f1a0d107a7, role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:53.842650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:53.846921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 66/136] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:18:53 2022] PUT /identity/v3/projects/6f8bf0894eb14221b351a195eb205a69/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:53.848523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fd06-2c3a-4b1f-98f8-a1e47f69b93a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.849009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fd06-2c3a-4b1f-98f8-a1e47f69b93a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.849602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57fd06-2c3a-4b1f-98f8-a1e47f69b93a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.851062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 67/137] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:53.865145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:53.865869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:53.866364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:53.876429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 71/138] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:18:53 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/users/9a6ac7756ddb4d96967796f1a0d107a7/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:53.889497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:53.891162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:53.956360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:53.975507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 68/139] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:54.014565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:54.147435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56609-96c1-417a-ab06-766cf194ba8a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:54.186629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:54.187857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:54.188352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:54.188827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:54.190424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:54.192278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e184f-5e0b-4cf5-9fbf-97c3aff1e978 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:54.194191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 72/140] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:18:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:54.200807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:54.309036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cbd0f-e985-4a5d-b22d-2155625734ce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:54.345554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:54.346598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:54.347085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:54.347554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:54.348460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:54.350190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b338ea-71a5-415d-af06-2fbdff116abd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:54.372094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 69/141] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:18:54 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:18:54.661507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984da96-2d31-43ca-8d4c-52ef32b1ab20 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:54.662130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984da96-2d31-43ca-8d4c-52ef32b1ab20 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:54.662611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984da96-2d31-43ca-8d4c-52ef32b1ab20 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:54.664788 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 73/142] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:54.674888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:54.675118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:54.675405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:54.699225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:54.700430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:54.754100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:54.788750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 70/143] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:54.813958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984da96-2d31-43ca-8d4c-52ef32b1ab20 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:54.851642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:54.958119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984da96-2d31-43ca-8d4c-52ef32b1ab20 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:55.000333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:55.001149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.001456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.001736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.014034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:55.021965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:55.023200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:55.024018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 74/144] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:55.029243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:55.158404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07542a-cc8a-4d7e-b816-725165bbefce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:55.197518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:55.198514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.198986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.199450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.200381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:55.200963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:55.203017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:55.217301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 71/145] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:55.223148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:55.332890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:55.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e10b5-4c11-4512-af45-2fdff3b1cb8c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:55.397122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:55.398033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.398315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.398594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] PATH_INFO: `/v3/users/0da7e48d096c4a8d8290d78e5d0a640b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.400335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] RBAC: Authorizing `identity:get_user(user_id=0da7e48d096c4a8d8290d78e5d0a640b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:55.401821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:55.403766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 75/146] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:55 2022] GET /identity/v3/users/0da7e48d096c4a8d8290d78e5d0a640b => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:55.414544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:55.469775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:55.553043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a176d37-5c2b-487e-8f18-b43075c674d5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:55.593667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:55.594538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.594827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.595118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.596354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:55.598007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18328a-87ce-40b4-92e6-787638e7522f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:55.599755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 72/147] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:55 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:55.606512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:55.732001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c01fb-6361-4d4e-9bac-9344af723601 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:55.772376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:55.773276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.773577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.773903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/users/0da7e48d096c4a8d8290d78e5d0a640b/roles/0ae26fe95f5c41a8a3424f304b954f9d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.778767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, user_id=0da7e48d096c4a8d8290d78e5d0a640b, role_id=0ae26fe95f5c41a8a3424f304b954f9d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:55.780225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5cf3c-b88c-42dc-b586-eb773973e7a5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:55.803563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 76/148] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:18:55 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/users/0da7e48d096c4a8d8290d78e5d0a640b/roles/0ae26fe95f5c41a8a3424f304b954f9d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:18:55.895654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fbab-a360-4e4e-87d6-24bdcf5a6340 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.895965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fbab-a360-4e4e-87d6-24bdcf5a6340 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.896220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535fbab-a360-4e4e-87d6-24bdcf5a6340 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.897842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 73/149] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:55.903823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:55.904111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:55.904376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:55.927457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:55.928684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:55.937832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:56.019140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 77/150] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:56.075224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535fbab-a360-4e4e-87d6-24bdcf5a6340 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:56.188360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535fbab-a360-4e4e-87d6-24bdcf5a6340 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:56.190779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:56.227237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:56.228040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.228324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.228598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.243064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:56.250830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:56.252951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:56.254970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 74/151] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:56.261382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:56.375537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ed78-79ea-4658-8f82-f6d5e9e70365 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:56.419568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:56.420254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.420534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.420806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.421588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:56.421898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:56.423284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867afc90-59b9-46cc-9ed7-9664ebd2db20 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:56.435866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 78/152] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:56.441678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:56.574143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede5d63-30c7-4a9f-910d-0f1736796641 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:56.604304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7a18c-3e01-4f83-b44c-deae13904ac0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.604628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7a18c-3e01-4f83-b44c-deae13904ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.604855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7a18c-3e01-4f83-b44c-deae13904ac0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.606553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 79/153] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:18:56.616831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:56.617758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.618050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.618331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] PATH_INFO: `/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.619457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.619566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] RBAC: Authorizing `identity:get_user(user_id=a1d6aa7bda7d44f4ab9f044e2d9e08a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:56.619753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.620012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.621024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:56.622965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 75/154] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:56 2022] GET /identity/v3/users/a1d6aa7bda7d44f4ab9f044e2d9e08a7 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:56.628437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:56.634326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:56.658445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:56.659648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:56.671283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:56.766980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 80/155] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:56.802646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:56.805618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c8dcb-2778-4d33-9be9-237602679281 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:56.870027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:56.870685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.870974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.871255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.946808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ee289-ebc1-4e89-80b0-2c59c3b605e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:56.982712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:56.983508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:56.983787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:56.984059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:56.985960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:56.997230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:57.004876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:57.007602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:57.009780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 81/156] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:57.014247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:57.014247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:57.014522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:57.070137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:57.143464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccb6b4-9e9e-414d-af14-7204a8507a1d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:57.170764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 76/157] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:18:57.176799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:57.199492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:57.200197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.200479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.200756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.201539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:57.201846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:57.203256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:57.228232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 82/158] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:57.238375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:57.293914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a964acd-9966-4837-8a2f-9f8ab009fe13 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:57.337699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:57.344903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:57.345850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.346137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.346468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.350080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:57.351605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:57.358220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 77/159] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:57 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:57.363790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:57.387438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:57.469837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bb568-622e-45ec-863b-0426da1ed0bf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:57.535800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b39fd-3d4e-431f-9a34-094a978d0c73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:57.570306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:57.571793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.572080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.572358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.578233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:57.579722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:57.581730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 83/160] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:57 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:57.587282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:57.592571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:57.593294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.593608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.593891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.594750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:57.595067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:57.596520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d608413-c39e-4849-851f-a9418a97fcf5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:57.603390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 78/161] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:57 2022] GET /identity/v3/role_assignments?user.id=a1d6aa7bda7d44f4ab9f044e2d9e08a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:57.770073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1291-7144-432f-8565-f31d932fb6c5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:57.771510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5f0-82d0-4cb0-b81d-aef093393d77 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.771808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5f0-82d0-4cb0-b81d-aef093393d77 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.772091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ec5f0-82d0-4cb0-b81d-aef093393d77 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.774266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 79/162] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:57.779822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.780123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.780374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.817642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:57.818869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:57.820646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:57.844511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:57.845205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:57.845521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:57.845801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:57.972630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:57.992269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 80/163] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:58.004657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:58.006169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bd9c1-224d-4af4-a9cb-839b9d0a571d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:58.043927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.185854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 84/164] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:18:58.188422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4422b9-d468-4a6a-a455-d205b56d27ae None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.188718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4422b9-d468-4a6a-a455-d205b56d27ae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.189001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4422b9-d468-4a6a-a455-d205b56d27ae None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.192539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 85/165] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:58.193374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4422b9-d468-4a6a-a455-d205b56d27ae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.240633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9da8-1f1e-48f6-953a-add3176852f1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.249683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:58.314138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.314945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.315233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.315563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.334237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b3ab2f2d-b042-4a84-86ae-8baaca1c3b4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:58.343004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:58.343894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:18:58.344840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 81/166] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:18:58.346027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.346119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.348143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.358709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4422b9-d468-4a6a-a455-d205b56d27ae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.374224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:58.375470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:58.398638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.400071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.400363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.400703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.403206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:58.405111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:58.407179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 86/167] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:58.408122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.484572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:58.491165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 82/168] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:18:58.492083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.508895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:58.517887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea102c-8daa-448a-a177-3b8d3dd42741 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.556127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.556953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.557415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.557795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.558709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:58.559135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:58.560620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:58.569524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 87/169] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:58.570459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.661876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79797b-9719-4b0d-8ec9-54337033cabf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.679772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e7df-4996-43d0-8f50-a8c11c4fe8eb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.716058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.717025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.717504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.717886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.721346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.722073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.722362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.722641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.723481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:58.723792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:58.725249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:58.730558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:18:58.738732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:18:58.740870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:18:58.743192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:18:58.744656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 83/170] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=37d514b6250f41f1a7a945e02510a6df => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:58.744656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.744927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 88/171] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:18:58.749392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:58.777739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:58.840668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:58.923522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be859b9-d8c1-4955-94cf-ace6c2999697 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.948190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d9298-7e6a-4c5f-bc78-5cfad3b57e4f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:58.982726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:58.983618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:58.983905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:58.984181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:58.988376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:58.989893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:58.991824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 84/172] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:58.998228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.015571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.016262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.016549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.016828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.017631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:18:59.017942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.019396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.044973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 89/173] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:59.050429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.151693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973783d-fbfd-4d3a-8938-27a18236eb3f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.193596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.202919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.203947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.203947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.204143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.239575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.277471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8058d6-7612-465d-8deb-c96540b7c450 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.314919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.316075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.316511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.316931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] PATH_INFO: `/v3/users/9a6ac7756ddb4d96967796f1a0d107a7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.318537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] RBAC: Authorizing `identity:get_user(user_id=9a6ac7756ddb4d96967796f1a0d107a7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.320432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.322405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 90/174] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/users/9a6ac7756ddb4d96967796f1a0d107a7 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:18:59.329380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.334218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.335724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c50f2e-f85c-4926-9570-b0f8eafedd64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.450000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa67c2-de18-4798-bdd5-04876f7220eb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.487951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.488855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.489287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.489740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.512182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 85/175] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:18:59.513573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e1-b958-445f-8032-da98bf499bd6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.513866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e1-b958-445f-8032-da98bf499bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.514147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e1-b958-445f-8032-da98bf499bd6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.515434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 86/176] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:59.522688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.522984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.523262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.534344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.558355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:18:59.559569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:18:59.566995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.568813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.575363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.683298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 91/177] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:18:59.683643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 87/178] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:18:59.684520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.698957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.753496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.809416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fedea-dacb-441f-a465-db63b7c2ea04 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.825983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1d345-a2a7-47ac-9d2d-56e2aebe8213 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.852273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.853444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.853735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.854015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] PATH_INFO: `/v3/projects/6f8bf0894eb14221b351a195eb205a69` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.855654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=6f8bf0894eb14221b351a195eb205a69)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.857551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.859956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 88/179] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/projects/6f8bf0894eb14221b351a195eb205a69 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:59.860734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.870454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:18:59.871341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.871628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.871909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.873127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:18:59.874634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93c605-a3f0-4ab2-a76a-02564e9654bd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:18:59.876353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 92/180] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:18:59.877634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b67c9-0bce-418e-b1b6-34e5456fca60 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.877924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b67c9-0bce-418e-b1b6-34e5456fca60 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.878201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b67c9-0bce-418e-b1b6-34e5456fca60 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.881379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 93/181] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:59.881379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641a66-20a4-40ce-b3e4-263228ce8bd9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:18:59.881557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641a66-20a4-40ce-b3e4-263228ce8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:18:59.881853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42641a66-20a4-40ce-b3e4-263228ce8bd9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:18:59.883393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 94/182] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:18:59.884186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42641a66-20a4-40ce-b3e4-263228ce8bd9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:18:59.885215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:18:59.977521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8dec5-4b9c-418f-86ce-eb657f971ba5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:18:59.997405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42641a66-20a4-40ce-b3e4-263228ce8bd9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.010125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.010604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.011416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.011683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.011956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.022075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:00.030933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:00.031824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:00.032601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 89/183] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:00.033231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.035359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.036014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.036294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.036567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.037703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=6f8bf0894eb14221b351a195eb205a69, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:00.038015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.039491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8c2e4-5a48-41d4-a3d1-8f353d67bf5c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.046060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 95/184] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:18:59 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=6f8bf0894eb14221b351a195eb205a69&role.id=37d514b6250f41f1a7a945e02510a6df => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.047239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c712f-4df2-4ead-846c-193d82ff262a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.047545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c712f-4df2-4ead-846c-193d82ff262a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.047792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c712f-4df2-4ead-846c-193d82ff262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.065493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b51c712f-4df2-4ead-846c-193d82ff262a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:00.066719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b51c712f-4df2-4ead-846c-193d82ff262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:00.123730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.145745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 96/185] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:00.146855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.147143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.147418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.167827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898a1dd-5583-4bad-b4e3-ad5a83c298a7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.168386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:00.170235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:00.208706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.209926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.210215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.210493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.211399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=9a6ac7756ddb4d96967796f1a0d107a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:00.211717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.213184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4c449-91cc-4c4c-9b0e-9d6c3831a89c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.220649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 90/186] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/role_assignments?user.id=9a6ac7756ddb4d96967796f1a0d107a7&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.222479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d280-e348-41cd-baed-337c4de78498 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.222887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d280-e348-41cd-baed-337c4de78498 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.223173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33d280-e348-41cd-baed-337c4de78498 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.224829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 91/187] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:00.225654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33d280-e348-41cd-baed-337c4de78498 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.256581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 97/188] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:00.257434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.304885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.376451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33d280-e348-41cd-baed-337c4de78498 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.383010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18a442-cbbe-46f1-b845-29095638c9c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.420032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.420395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.420677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.420994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.421809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:00.422120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.425386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7f595-5f70-4eae-92de-e013245b8218 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.430671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.433770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.434563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 92/189] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.434676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.434855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.435143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.436331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.436594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.437007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.437457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.438001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.439666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 98/190] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.440476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.457492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:00.458693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:00.526161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 93/191] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:00.527012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.553767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bd64a-264b-4cab-9e53-9f94611de0e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.556075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.587411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.588219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.588496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.588773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.598546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:00.606682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:00.607643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 99/192] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:00.608581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.627537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6514e-8ea2-4a74-8860-d455141cdd19 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.655681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.661738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.662606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.662885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.663156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.676532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.678012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.679684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 94/193] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.680972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.712413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8a615-ceb3-4921-9a8b-0ee7d2f72126 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.744930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.745620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.745898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.746171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.747486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.748951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.781469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 100/194] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:00 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:00.781469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.788501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b81ee-b829-4b79-b513-a42dacbd3881 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.801382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.831161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.832011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.832338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.832594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.844763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:00.853701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:00.854840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 95/195] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:00.855804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.899136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bbe4-9e9c-453c-90d6-edc9e3ed9de4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:00.908782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:00.937586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:00.938280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:00.939168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:00.939168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:00.939588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:00.939899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:00.941285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:00.951722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 101/196] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:00.953014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:00.973946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3f591-058b-474c-9035-70228bf58e64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.022791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.023599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.024053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.024431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.059695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.079082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cc111-8ed5-4403-b870-45b07c868fea None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.118578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.120094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.121288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.122118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.122499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.122880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.123734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:01.124176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.126331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.135104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 102/197] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:01.135104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.175613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.233453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 96/198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:01.234460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.268804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff47d3-b598-4e1c-845d-b535f8d4af9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.316582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.317565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.317949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.318324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.320182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.322358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.324889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 103/199] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:01.326033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.346184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.356530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb9475-9891-44b7-a6b8-b489c9463295 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.396212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.398470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.399491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.399969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] PATH_INFO: `/v3/users/0da7e48d096c4a8d8290d78e5d0a640b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.401971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=0da7e48d096c4a8d8290d78e5d0a640b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.404006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.406194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 97/200] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/users/0da7e48d096c4a8d8290d78e5d0a640b => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:01.407164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.437905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.462284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111275d-6e75-49a0-b32a-eb60530b9cb5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.497117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.498066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.498330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.498717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.499899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.501359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.503149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 104/201] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:01.503898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.567460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bccaba-63ba-4069-92d3-28718aeb5ce3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.607988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.618219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.619474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.620188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.620658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.630570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435680e-b0d4-4476-9973-f7d3d9fefd33 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.668255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.669061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.670188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.670188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.709729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.712218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.729970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.761393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.765068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.819460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 98/202] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:01.825499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.861105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.882493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 105/203] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:01.884213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.931914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa3c00-9559-4d42-898b-ea90e4959d79 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:01.951453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:01.971864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:01.972768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:01.973056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:01.973354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:01.974362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=9569cf96bee747c8a423977d6a5dd056) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:01.974689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:01.976455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:01.984654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 99/204] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/role_assignments?user.id=9569cf96bee747c8a423977d6a5dd056&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:01.985695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:01.993466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85de916-1d70-4e11-bd3f-aecdf37e15b0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:02.081279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:02.082723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:02.084719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:02.086746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:02.089463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:02.091411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:02.095352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:02.102076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:02.175608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:02.178247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfceeb0-e98b-4230-b994-aeef64a58fb4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:02.238663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:02.239914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:02.240399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:02.240875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:02.243116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:02.245500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:02.248471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 100/206] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:01 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:02.252097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2ce7-cce9-49b5-9564-fa12253a898b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:02.255078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:02.311118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:02.365628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:02.366424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:02.366635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:02.366916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:02.367837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=9569cf96bee747c8a423977d6a5dd056) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:02.368139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:02.369669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bc97c-d47d-4b75-ab31-46d9aa27e6e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:02.370410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdd324-be70-4fb2-aa49-f2276a15bce6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:02.393711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 107/207] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:02 2022] GET /identity/v3/role_assignments?user.id=9569cf96bee747c8a423977d6a5dd056&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:02.429402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:02.430255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:02.430666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:02.431044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:02.431998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, scope.project.id=9591568cf2904986aa8282568cb6b37b, user.id=0da7e48d096c4a8d8290d78e5d0a640b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:02.432415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:02.434002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58461a15-a55a-4c94-ba00-bef33a123b86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:02.440662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 101/208] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:02 2022] GET /identity/v3/role_assignments?user.id=0da7e48d096c4a8d8290d78e5d0a640b&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:03.143865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449daa-4225-4158-9115-cbb054d6c7a2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.144888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449daa-4225-4158-9115-cbb054d6c7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.145424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449daa-4225-4158-9115-cbb054d6c7a2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.147028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 108/209] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:03.152623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.153169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.153654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.156492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2ea2-ec4d-41e7-ba89-2eefe8a2a64d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.156773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2ea2-ec4d-41e7-ba89-2eefe8a2a64d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.157045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2ea2-ec4d-41e7-ba89-2eefe8a2a64d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.158526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 109/210] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:03.165982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.166592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.167106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.176596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:03.177872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:03.189159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:03.191413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:03.253449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 102/211] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:03.278087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.292808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 110/212] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:03.316289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.330640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:03.362466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:03.407915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ada06-752f-475c-b1fe-b2dfd2164de6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:03.426707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ab84f-6052-455e-8244-b546ddc5fbda None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:03.451702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:03.453565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.454014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.454429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.456182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:03.458210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:03.460537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 103/213] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:03.467083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.473221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:03.473663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.473946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.474221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.488069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:03.498887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:03.499759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:03.505611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 111/214] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:03.507705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.595780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb3437-2eb3-47f3-904e-5b8728b7955a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:03.609682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:03.613313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fec2-fd33-40bc-8899-8ef8f4c588b4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:03.615208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:03.644230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:03.645358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.645641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.645919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.646914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:03.649221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:03.655113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:03.656189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.656669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.657143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.658216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:03.658740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:03.660403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:03.681359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 104/216] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:03 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:03.681558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 112/216] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:03.686182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.807029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4c39-af27-4188-ace2-cef1053e6a4f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:03.843725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:03.844766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:03.845048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:03.845346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] PATH_INFO: `/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:03.859710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] RBAC: Authorizing `identity:get_user(user_id=718e8a1ee7bd4856b09cac3ab4db7f73)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:03.861156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31127e-a900-4d7c-a4a2-b2a678b11833 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:03.867254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 105/217] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:03 2022] GET /identity/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:03.869024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:03.982368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cd08a-9797-4456-a6d1-0e6c4c9989af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:04.020826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:04.021512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.021851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.022128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.060326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:04.099981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:04.101481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:04.131282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1d5a-3be6-4cc5-bc29-fe74fdb03e2c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.131591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1d5a-3be6-4cc5-bc29-fe74fdb03e2c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.131836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1d5a-3be6-4cc5-bc29-fe74fdb03e2c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.135397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 106/218] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:04.141836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.142112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.142393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.165434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:04.166630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:04.192265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:04.230577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 113/219] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:19:04.241705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:04.276462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 107/220] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:04.300153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:04.373710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:04.417004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dac9d6-39fa-4714-aece-5ed620170e19 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:04.453203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:04.454129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.454555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.454837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.456127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:04.457668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:04.459760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 114/221] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:04.465996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:04.508094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e752a2b-e9f4-41bb-a896-80988a024086 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:04.548045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:04.548900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.549182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.549699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.562507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:04.571213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:04.572078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:04.572853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:04.577902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 108/222] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:04.579320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:04.613701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:04.642651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35902a9-6fd1-4b2e-8c20-13397608c311 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:04.686596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:04.696180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:04.697358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.697652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.697934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.699059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=718e8a1ee7bd4856b09cac3ab4db7f73) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:04.699628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:04.701506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ffba2-57df-4bf7-a5e9-3cf17c7e4516 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:04.709444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 115/223] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/role_assignments?user.id=718e8a1ee7bd4856b09cac3ab4db7f73&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=37d514b6250f41f1a7a945e02510a6df => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:04.710764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaa398-c413-4e0f-81eb-9d5e3e0cdec9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.711080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaa398-c413-4e0f-81eb-9d5e3e0cdec9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.711390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaa398-c413-4e0f-81eb-9d5e3e0cdec9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.712650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 116/224] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:04.725130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01165ebf-5e1e-4b1c-802d-b61e33f8fbc7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:04.731953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.732289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.732480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.765417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:04.766614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:04.797749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:04.798494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:04.798782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:04.799114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:04.799914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:04.801071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:04.802758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:04.814398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 109/225] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:04.819570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:04.917747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 117/226] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:04.946411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.012095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bfe68-bf3b-4ef4-9fba-379b7c8f733f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.015538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:05.049809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.051417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.051904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.052397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.053856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.056214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.058761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 110/227] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:05.066388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.125771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd931d-96b8-45c9-b3dc-003083572115 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.156293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:05.166629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.167457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.167738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.168005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.189386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:05.205931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:05.206880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:05.207645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 118/228] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:05.215293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.269077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92590bb6-4250-4a2e-8da6-85f066b8a300 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.332726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.336409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.336695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.336976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.339502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.342300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.346975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 111/229] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:05 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:05.354273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.379762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:05.403367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd40b0d-19f1-4f0e-b783-a8b07cd408cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.490010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087e03-7634-4a4c-8fb8-346513a46950 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.527010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.528685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.529179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.529741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/9569cf96bee747c8a423977d6a5dd056/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.532002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, user_id=9569cf96bee747c8a423977d6a5dd056, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.533752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.538043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.538732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.539018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.539829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.540640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:05.540951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.545389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.554129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 119/230] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:05.560042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.566635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 112/231] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:05 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/9569cf96bee747c8a423977d6a5dd056/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:05.619048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:05.674245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2d11-1b33-498b-bdad-0f30a4e8fe63 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.719961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.721158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.721614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.722034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.723507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.725176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.727152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 120/232] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:05 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:05.733062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:05.846226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:05.878636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198373ea-c47c-4541-b2a8-e18bedd4f301 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:05.923541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:05.924408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:05.924690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:05.924964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:05.926441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:05.927998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f29a82-84ab-4f07-8cc1-0a6ff8ddb35a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:05.930744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 113/233] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:05 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:05.936666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:06.064903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48939352-21c2-47a7-957c-c4b4e5a5338a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:06.085690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:06.100338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:06.101157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.101469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.102284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/9569cf96bee747c8a423977d6a5dd056/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.104624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, user_id=9569cf96bee747c8a423977d6a5dd056, role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:06.106124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040a9e5-20f1-491b-be46-5a7afe2ccb9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:06.123232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 121/234] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:05 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/9569cf96bee747c8a423977d6a5dd056/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:06.319295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708665-8327-4ebf-be93-5a1096eb5dc9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.319584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708665-8327-4ebf-be93-5a1096eb5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.319862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e708665-8327-4ebf-be93-5a1096eb5dc9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.321833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 114/235] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:06.327213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.327505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.327780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.347729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:06.349265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:06.412325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b940a-6ab3-4f48-b179-8a855e1c4c5a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.412325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b940a-6ab3-4f48-b179-8a855e1c4c5a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.412325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b940a-6ab3-4f48-b179-8a855e1c4c5a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.412325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 115/236] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:06.417498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.417791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.418121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.445484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 122/237] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:06.458372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:06.459597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:06.466440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:06.536939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 116/238] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:06.562234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:06.640852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:06.678864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c0290-d784-4e6f-b776-8ff48eda2a26 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:06.723958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:06.724984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.725270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.725573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.729434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:06.731448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:06.733419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 117/239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:06.738811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:06.741438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a370b31-8210-48e5-8a62-1e4fe450080a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:06.771390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:06.823240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:06.824093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:06.824381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:06.824652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:06.847584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:06.860048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:06.860954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:06.862189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 123/240] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:06.868711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:06.956621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeb328-826f-4a22-8679-d603a83af514 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:06.981188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c161f66-39ea-457c-a2e6-c009bf612f66 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.008762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.009786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.010286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.010779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.011669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.013383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c149fe-a781-4790-8fdf-10dfde91d9f0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.022700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.026782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.027831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.028067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.028480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.029272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:07.029736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.031404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.039754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 118/241] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:06 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:07.041590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94660-5a12-4bb8-a561-0520bf03e3f2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.042186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94660-5a12-4bb8-a561-0520bf03e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.042658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e94660-5a12-4bb8-a561-0520bf03e3f2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.044437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 119/242] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:07.045478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 124/243] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:07.055920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.056599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.057104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.071519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.095552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:07.099102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:07.153737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.252103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 120/244] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:07.266467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70ab0-3539-4123-896b-3d048e6a8d21 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.273211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.328522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.329608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.329895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.330159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] PATH_INFO: `/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.348104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] RBAC: Authorizing `identity:get_user(user_id=e41fac4291af4b39aafeb8fc7d0a1d51)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.349710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.351772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 125/245] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:07.358352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.392337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34717b-401e-4f1b-92b7-bec98a37a9fb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.419632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.448416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.449226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.449539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.449820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.453793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.470238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadf99a-468a-4aa8-b98d-d55aceedfa1e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.470440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:07.479032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:07.479914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:07.481218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 121/246] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:07.491263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.518009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.518599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.518880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.519160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.618106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.621025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.643264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3c8-f235-46b8-a417-d53f8113d1ea None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.692857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.694259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.695140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.696124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.697361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:07.697891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.699772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.709766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 122/247] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:07.718183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.735335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.761076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 126/248] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:07.767113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:07.807757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:07.898695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d8d0-3af9-4203-957d-e90e66b2c959 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.928439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624b7d5-53ad-4f0b-a95a-0e5baa7cd908 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:07.935957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:07.936912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:07.937197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:07.937496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:07.938685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:07.940153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:07.942334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 127/249] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:07.948692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:08.009895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:08.034401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:08.035236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.035594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.035841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] PATH_INFO: `/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.037156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] RBAC: Authorizing `identity:get_user(user_id=718e8a1ee7bd4856b09cac3ab4db7f73)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:08.045465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:08.047635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 123/250] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73 => generated 297 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:08.058411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:08.065377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec181-de12-4019-9e09-15a2d3033088 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:08.103616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:08.104294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.104581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.104856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.105725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=e41fac4291af4b39aafeb8fc7d0a1d51) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:08.106038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:08.107495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:08.113164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 128/251] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:07 2022] GET /identity/v3/role_assignments?user.id=e41fac4291af4b39aafeb8fc7d0a1d51&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:08.247357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003f6a5-df89-4b5f-a613-3b61c63b9272 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:08.271975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:08.301234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:08.302609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.303118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.303622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.305716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:08.307391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c82b-ac53-411e-9c49-185519eeb0d2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:08.309455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 124/252] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:08 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:08.318463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:08.450161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f8ad1-889c-4d98-8054-115a97a3c84b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:08.451378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420020-e779-40b7-9202-8dee2e999521 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.451867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420020-e779-40b7-9202-8dee2e999521 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.452378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59420020-e779-40b7-9202-8dee2e999521 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.457452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 125/253] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:08.459177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6d-4cdf-4b1e-9efc-1e34057bbe42 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.459469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6d-4cdf-4b1e-9efc-1e34057bbe42 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.459749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f6d-4cdf-4b1e-9efc-1e34057bbe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.478198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a98d6f6d-4cdf-4b1e-9efc-1e34057bbe42 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:08.480264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:08.480999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a98d6f6d-4cdf-4b1e-9efc-1e34057bbe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:08.514422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352572a7-3997-422c-896c-058116b520ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:08.515390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352572a7-3997-422c-896c-058116b520ee None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.515671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352572a7-3997-422c-896c-058116b520ee None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.515995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352572a7-3997-422c-896c-058116b520ee None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/718e8a1ee7bd4856b09cac3ab4db7f73/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.521026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352572a7-3997-422c-896c-058116b520ee None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, user_id=718e8a1ee7bd4856b09cac3ab4db7f73, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:08.522580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352572a7-3997-422c-896c-058116b520ee None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:08.546563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 129/254] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:08 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/718e8a1ee7bd4856b09cac3ab4db7f73/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:08.596733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 126/255] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:08.636602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352572a7-3997-422c-896c-058116b520ee None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:08.776165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352572a7-3997-422c-896c-058116b520ee None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:08.823942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:08.824775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.825059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.825385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.833909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d74103-f2ba-42f9-bffe-6adb0d536439 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.834819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d74103-f2ba-42f9-bffe-6adb0d536439 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.835291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d74103-f2ba-42f9-bffe-6adb0d536439 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.837434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 127/256] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:08.844043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:08.844785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:08.845257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:08.852428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:08.860970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:08.861920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:08.863054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 130/257] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:08.868446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:08.882366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:08.883646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:08.932971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:09.016477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 128/258] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:09.058451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12028fbd-11a8-4b33-b856-b8e6130ebe68 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.060382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.132140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.132758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.133599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.133876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.134642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:09.134952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:09.136368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:09.146762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:09.153099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 131/259] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:09.159129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.262533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4cae2-0e1f-41b1-92f3-1b47d120f670 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.327193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.328443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.331603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.332092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.351342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:09.352618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:09.363603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e533f1-a8af-44e6-ab86-38a0c4326d43 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:09.371219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:09.372087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:09.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:09.377734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 129/260] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:09.385097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.442031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.442918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.443207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.443473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.447686 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:09.449184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:09.451032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 132/261] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:09.457583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.499228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e670625-89f9-4d5d-b4e0-87678bc7fb27 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.504397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:09.573005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.574072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.574722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.576010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.577643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:09.578306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:09.580903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:09.595982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 130/262] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:09.602846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.657325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c932e0-bcf5-4972-8b06-585928600c9a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.721383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.722465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.722974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.723575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.790914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a7d3b-eece-40e1-be9c-04915d27146c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:09.803999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:09.841068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:09.842714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:09.843196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:09.843667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] PATH_INFO: `/v3/users/9569cf96bee747c8a423977d6a5dd056` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:09.845122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9569cf96bee747c8a423977d6a5dd056)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:09.847000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:09.849271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 131/263] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/users/9569cf96bee747c8a423977d6a5dd056 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:09.855445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:09.867978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:09.869871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:09.917790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.034328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000e0c-5e49-4de1-a517-76b9fe7002d9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:10.048723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 133/264] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:10.057163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:10.103130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:10.104060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.104349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.104626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.168132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.219892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d656d-ed25-446e-ab72-d6e5a3fdd723 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:10.235246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:10.236730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866310eb-fa61-4a05-af6e-9a7e99c2d642 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:10.305502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:10.305869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.306970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.307253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.307544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.308855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:10.310384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:10.312446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 134/265] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:10 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:10.318095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:10.435728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 132/266] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:10.436987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19826-a662-4231-8f43-d428b308f9e9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.437268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19826-a662-4231-8f43-d428b308f9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.437567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19826-a662-4231-8f43-d428b308f9e9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.438958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 133/267] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:10.446251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19826-a662-4231-8f43-d428b308f9e9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:10.533641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4905f-076f-4fc0-b39b-fa637a9b36c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:10.561372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19826-a662-4231-8f43-d428b308f9e9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:10.593372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.598873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:10.599049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:10.599609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.599893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.600176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.600518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.600977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.601100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=9569cf96bee747c8a423977d6a5dd056) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:10.601311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:10.602181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.603326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:10.603684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:10.605266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb93203-0efb-47a9-bae2-76ca303667b9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:10.607453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 134/268] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:10 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:10.608933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19061dfd-348d-4069-a138-9f694873c45b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.609346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.610848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19061dfd-348d-4069-a138-9f694873c45b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.611380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19061dfd-348d-4069-a138-9f694873c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.617074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 135/269] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:10 2022] GET /identity/v3/role_assignments?user.id=9569cf96bee747c8a423977d6a5dd056&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:10.618229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:10.644148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-19061dfd-348d-4069-a138-9f694873c45b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:10.645420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-19061dfd-348d-4069-a138-9f694873c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:10.794827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561a664-8ea9-484a-bc32-88e56835fad7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:10.831567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:10.832490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.832913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.833381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.834362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=9569cf96bee747c8a423977d6a5dd056) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:10.835128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:10.836974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd04fa4-4d8b-4c73-91ed-cf7ab1e94d4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:10.845030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 136/270] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:10 2022] GET /identity/v3/role_assignments?user.id=9569cf96bee747c8a423977d6a5dd056&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:10.847282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078ffe7-0832-4aa2-8100-8f5bed796ca8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.847829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078ffe7-0832-4aa2-8100-8f5bed796ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.848246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078ffe7-0832-4aa2-8100-8f5bed796ca8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.849962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 137/271] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:10.859461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c7e1-b68b-40f9-a37c-da6fbc45037a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.860186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c7e1-b68b-40f9-a37c-da6fbc45037a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.860638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651c7e1-b68b-40f9-a37c-da6fbc45037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.874792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 135/272] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:10.885229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2651c7e1-b68b-40f9-a37c-da6fbc45037a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:10.886801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2651c7e1-b68b-40f9-a37c-da6fbc45037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:10.920514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19061dfd-348d-4069-a138-9f694873c45b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:10.966277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:10.979315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 138/273] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:10.980991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8e98-325f-47f4-86b9-20945807d0e5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.981468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8e98-325f-47f4-86b9-20945807d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.981887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f8e98-325f-47f4-86b9-20945807d0e5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:10.984482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 139/274] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:10.990325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:10.990734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:10.991229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.010936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:11.012784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:11.073417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19061dfd-348d-4069-a138-9f694873c45b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.094210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 140/275] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:11.095582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.108375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.116540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.118318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.118935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.119399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.120818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.122542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.124457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 136/276] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:11.130819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.217284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0188-582b-4384-aced-ffa5045180d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.244312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.256300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.257103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.257473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.257769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.261843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee42a0-c790-4d35-8a1e-6b02a5b923c8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.268690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:11.276910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:11.277868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:11.278735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 141/277] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:11.279601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.307332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.307999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.308280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.308554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.309268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.310724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.340273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 137/278] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:11 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:11.341416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.405777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.433789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f9665-ae29-4aee-b189-4b127ced3481 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.478354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.479563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.480046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.480398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52929bb8-9235-447f-9d44-f9f69997199d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.480731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.493978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:11.502378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:11.503287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:11.504067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:11.506804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 142/279] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:11.512408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.527962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.528634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.528915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.529191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.529971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:11.530298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.531657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.548518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 138/280] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:11.549813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.575116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.647746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de2036-a318-4bad-a36e-19d94d9bd02e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.683599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f6db6-da4b-4214-8705-7c17c2545be7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.699940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.700632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.700919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.701199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.701996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:11.702309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.703715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae205c-b57b-4493-9bb1-06a28123180c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.706032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.716768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 143/281] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:11.718796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d12-a068-45b2-99a2-675d130f2a68 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.719051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d12-a068-45b2-99a2-675d130f2a68 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.719330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf36d12-a068-45b2-99a2-675d130f2a68 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.720704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 144/282] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:11.725782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36d12-a068-45b2-99a2-675d130f2a68 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.726633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.727788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.728263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.729053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] PATH_INFO: `/v3/users/02a36ca27cfb47398966b590635e8c57` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.751674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] RBAC: Authorizing `identity:get_user(user_id=02a36ca27cfb47398966b590635e8c57)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.753525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f75b3-83e7-4b38-bebe-2f74fbfa8465 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.755369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 139/283] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/users/02a36ca27cfb47398966b590635e8c57 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:11.756471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.756761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.757038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.775714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:11.776748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:11.849716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 140/284] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:11.850639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.855185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf36d12-a068-45b2-99a2-675d130f2a68 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.863731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.896468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.897495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.897933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.898082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] PATH_INFO: `/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.900161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] RBAC: Authorizing `identity:get_user(user_id=e41fac4291af4b39aafeb8fc7d0a1d51)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:11.901898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:11.904157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 145/285] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:11.909561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:11.956881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c390a3-7af6-457d-a619-6fde8476a545 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:11.960430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:11.995095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:11.996078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:11.996564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:11.997035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:11.998091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, user.id=02a36ca27cfb47398966b590635e8c57, scope.system=all) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:11.998615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.000306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:12.005957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 141/286] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/role_assignments?user.id=02a36ca27cfb47398966b590635e8c57&scope.system=all&role.id=37d514b6250f41f1a7a945e02510a6df => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:12.006981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:12.046774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a458ec0-a8a9-4935-9809-e1b155da21cd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.106569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.107384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.107670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.107948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.129359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:12.131723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:12.132541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:12.134198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 146/287] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:12.142536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:12.185726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:12.188444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69201e40-4b66-4855-9502-12bf58c48c7a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.273275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.274458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.274744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.275031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.278858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.279054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:12.282082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 142/288] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:12 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:12.290779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:12.291892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3be2d4-8cf1-45a7-8f45-4f341b86e596 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.332004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.334328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:12.335302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.335775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.336279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.337290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:12.337820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.339426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:12.354089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 147/289] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:12.360902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:12.470914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2e5de-aab2-4885-b3a7-b33751f2184f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.518429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:12.533867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c37c6-e032-40a8-88c2-405825055d34 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.545406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.546305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.546776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.547060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/e41fac4291af4b39aafeb8fc7d0a1d51/roles/0ae26fe95f5c41a8a3424f304b954f9d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.551156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, user_id=e41fac4291af4b39aafeb8fc7d0a1d51, role_id=0ae26fe95f5c41a8a3424f304b954f9d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.552600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:12.578360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 143/290] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:12 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/users/e41fac4291af4b39aafeb8fc7d0a1d51/roles/0ae26fe95f5c41a8a3424f304b954f9d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:19:12.598541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.599430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.599768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.600055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] PATH_INFO: `/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.601300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] RBAC: Authorizing `identity:get_user(user_id=718e8a1ee7bd4856b09cac3ab4db7f73)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.602841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:12.605888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 148/291] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:12 2022] GET /identity/v3/users/718e8a1ee7bd4856b09cac3ab4db7f73 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:12.610461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:12.752615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:12.759566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b5f3-283b-4cec-ac2e-8d97336b3997 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:12.816572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:12.817650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:12.818131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:12.818596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:12.904253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:12.905753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:13.004979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:13.017460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 144/292] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:13.019516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:13.152396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978763a-238a-4c55-9ca8-ea4f5b65ccec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:13.199364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:13.200240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.200536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.200793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.202026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:13.203489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef44b44-3034-40c0-9eda-7c86f7373013 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:13.205365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 149/293] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:13 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:13.211977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:13.329921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9bd0b-6726-4a4d-acd5-5a7d22f2c57f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:13.373097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:13.373786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.374072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.374353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.375184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=718e8a1ee7bd4856b09cac3ab4db7f73) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:13.375498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:13.376931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24a6aa-5926-4842-ad4d-d90b95129fbf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:13.389472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 145/294] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:13 2022] GET /identity/v3/role_assignments?user.id=718e8a1ee7bd4856b09cac3ab4db7f73&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=37d514b6250f41f1a7a945e02510a6df => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:13.860100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1897cab-57d0-4ff6-9292-35c1eeacb7f0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.860385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1897cab-57d0-4ff6-9292-35c1eeacb7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.860821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1897cab-57d0-4ff6-9292-35c1eeacb7f0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.862646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 150/295] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:13.868449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.868987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.869581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.887485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:13.890301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:13.940794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001032d-fd68-4ed7-9e91-7cb9ccec5568 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.941078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001032d-fd68-4ed7-9e91-7cb9ccec5568 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.941593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001032d-fd68-4ed7-9e91-7cb9ccec5568 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.943105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 151/296] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:13.948773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:13.949275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:13.949778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:13.966222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:13.972118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:13.974230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:13.989838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 146/297] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:14.003625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:14.011168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.059277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 152/298] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:14.082713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.130776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2bc16-57bf-45a8-aa57-5b91e882a252 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.170606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.171521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.171811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.172089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.183740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:14.192129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:14.193071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:14.193896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:14.198998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 147/299] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:14.199950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bd5db-6fe0-4844-8617-66c16e0d8af7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.200639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.246868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.248488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.249370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.250146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.251868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:14.254368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:14.256221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:14.261874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 153/300] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:14.264679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.305953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:14.318135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad273-c37c-4b28-985c-255018a4f5d7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.358819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.359485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.359772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.360050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.360801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:14.361109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:14.362709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:14.376235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 148/301] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:14.377705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.396163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71df86-7e0f-49bf-baf9-c615c2adedbc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.451741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.452749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.453167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.453751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.454642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:14.456207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:14.476330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 154/302] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Jul 12 14:19:14 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:19:14.494070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da344a5-6673-444a-89a3-ae28d91a8c27 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.511032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:14.537160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.538099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.538449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.538729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] PATH_INFO: `/v3/users/e58593246fa743a79b081fce11a0140f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.556870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] RBAC: Authorizing `identity:get_user(user_id=e58593246fa743a79b081fce11a0140f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:14.558572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c90872-025c-4a97-a0b1-3bf6d39c91f1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:14.564391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 149/303] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/users/e58593246fa743a79b081fce11a0140f => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:14.570575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.681723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:14.694839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f796a0-e957-41a8-9313-53f53f2d4654 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:14.731235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:14.731903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.732191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.732471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.733314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, user.id=e58593246fa743a79b081fce11a0140f, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:14.733666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:14.735142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b4838-a273-459b-9d95-1df4dccd886f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:14.742924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 155/304] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/role_assignments?user.id=e58593246fa743a79b081fce11a0140f&scope.system=all&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:14.768680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8786-01a5-4655-8697-bf7c5eef8420 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.769029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8786-01a5-4655-8697-bf7c5eef8420 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.769320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e8786-01a5-4655-8697-bf7c5eef8420 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.770816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 150/305] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:14.776875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5a56d-9c43-49c3-84bd-ccce2ff954f2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.777197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5a56d-9c43-49c3-84bd-ccce2ff954f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.777491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5a56d-9c43-49c3-84bd-ccce2ff954f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.796183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-02f5a56d-9c43-49c3-84bd-ccce2ff954f2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:14.797509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02f5a56d-9c43-49c3-84bd-ccce2ff954f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:14.888191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 156/306] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:14.910653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e8786-01a5-4655-8697-bf7c5eef8420 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:14.994565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d187c-e5f6-4551-82f6-87b8c5747ee2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:14.995198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d187c-e5f6-4551-82f6-87b8c5747ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:14.995693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d187c-e5f6-4551-82f6-87b8c5747ee2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:14.997498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 157/307] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:15.002762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.003049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.003315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.021963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:15.023957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:15.025892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e8786-01a5-4655-8697-bf7c5eef8420 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.064177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.065192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.065606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.065973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.077775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:15.086315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:15.087480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:15.088440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:15.089028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 151/308] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:15.094779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.111111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 158/309] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:15.127661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:15.130028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.145402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:15.207086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b784106-5310-4054-8166-9c84872fee8c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.249750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5a788-12fa-45e0-91dd-e1ba19f5bfc3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.250647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.251326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.251612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.251893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.252642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:15.252949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.254352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.268457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 152/310] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:15.269729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.297268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.298132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.298413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.298688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.311028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:15.318865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:15.319774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:15.321114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 159/311] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:15.326826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.388771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:15.399790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d0eaa-4df4-4501-9db0-63b6362c5eee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.407032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:15.433999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726172ab-db95-447d-8bbe-4222eae4d409 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.441369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.442528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.443100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.443572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] PATH_INFO: `/v3/users/02a36ca27cfb47398966b590635e8c57` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.445054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] RBAC: Authorizing `identity:get_user(user_id=02a36ca27cfb47398966b590635e8c57)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.446763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.449093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 153/312] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/users/02a36ca27cfb47398966b590635e8c57 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:15.455109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.477260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.478327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.478840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.479847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.481100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:15.481650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.484199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.496932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 160/313] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:15.502600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.569968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb90fa-5012-49b8-abdc-18f9a0740aee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.606174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e691518-966b-42db-a56f-834030e68324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.607374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e691518-966b-42db-a56f-834030e68324 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.607886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e691518-966b-42db-a56f-834030e68324 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.608362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e691518-966b-42db-a56f-834030e68324 None admin] PATH_INFO: `/v3/system/users/02a36ca27cfb47398966b590635e8c57/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.610170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e691518-966b-42db-a56f-834030e68324 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=02a36ca27cfb47398966b590635e8c57, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.610822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd939602-ed1c-4cb8-86ec-5810d70c9d29 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.611944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e691518-966b-42db-a56f-834030e68324 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.629498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 154/314] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Jul 12 14:19:15 2022] PUT /identity/v3/system/users/02a36ca27cfb47398966b590635e8c57/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:19:15.660008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.663723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.664593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.664875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] PATH_INFO: `/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.666905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] RBAC: Authorizing `identity:get_user(user_id=e41fac4291af4b39aafeb8fc7d0a1d51)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.668585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.670721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 161/315] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/users/e41fac4291af4b39aafeb8fc7d0a1d51 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:15.676561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e691518-966b-42db-a56f-834030e68324 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:15.714036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:15.811414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e691518-966b-42db-a56f-834030e68324 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:15.854129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:15.855112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:15.855603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:15.856095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:15.933859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:15.935700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:15.961685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:16.043874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 155/316] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:16.045990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:16.059719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:16.156355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5f5ae-8c62-4a55-93dc-afc2fb0da37d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:16.196347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:16.197218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.197526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.197801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.198975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:16.200413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a810f08-4b47-4238-a1e9-dea63f08e8e2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:16.202518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 162/317] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:16 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:16.208287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:16.312914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ccf41-5175-4f75-9314-c9b949d3d415 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:16.337976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:16.350347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:16.351385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.351919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.352398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.353480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, scope.project.id=f49c87f5ecd84589b554ef46f87ef734, user.id=e41fac4291af4b39aafeb8fc7d0a1d51) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:16.353995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:16.355638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8dd09-1f6e-498b-a7a2-a6f812a0752f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:16.369464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 156/318] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:16 2022] GET /identity/v3/role_assignments?user.id=e41fac4291af4b39aafeb8fc7d0a1d51&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:16.673173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d3bc-8916-451f-bfdf-9b957a4badf3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.673431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d3bc-8916-451f-bfdf-9b957a4badf3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.673720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433d3bc-8916-451f-bfdf-9b957a4badf3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.675200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 163/319] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:16.680856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d00f2-585b-4b66-b01e-bd006f59a565 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.681146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d00f2-585b-4b66-b01e-bd006f59a565 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.681453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d00f2-585b-4b66-b01e-bd006f59a565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.698294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4c3d00f2-585b-4b66-b01e-bd006f59a565 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:16.699778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4c3d00f2-585b-4b66-b01e-bd006f59a565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:16.769531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 157/320] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:16.791984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433d3bc-8916-451f-bfdf-9b957a4badf3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:16.849501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aaa9e-ad48-454c-a757-b451c446b815 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.849825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aaa9e-ad48-454c-a757-b451c446b815 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.850073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aaa9e-ad48-454c-a757-b451c446b815 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.851487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 158/321] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:16.856487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.856787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.857041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.869062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:16.873390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:16.874808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:16.905246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433d3bc-8916-451f-bfdf-9b957a4badf3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:16.943126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:16.943881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:16.944153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:16.944438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:16.947200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:16.947713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:16.949232 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 159/322] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:16.949608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 164/323] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:16.959051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:16.972861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:17.027055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:17.062560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa62f0-305e-4636-a996-16e3b03befeb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:17.083075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ca242-d642-43dc-96fb-a32ba288e4b0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:17.099849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:17.100609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.100965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.101576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.102102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:17.103545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:17.114514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:17.128933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 165/324] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Jul 12 14:19:16 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:19:17.129133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:17.129133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.129133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.129133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.140121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:17.148085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:17.149609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:17.150439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:17.155958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 160/325] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:17.157089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:17.266147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bcd26-bd0f-4744-ab53-337ac197963f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:17.301371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:17.302292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.302728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.303194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.304152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:17.304950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:17.306456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:17.315328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 166/326] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:17.321027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:17.434801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e41be-9b04-4b76-a4fe-4c3cf246b982 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:17.447350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:17.477722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:17.478997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.479493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.479952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] PATH_INFO: `/v3/users/e58593246fa743a79b081fce11a0140f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.481573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] RBAC: Authorizing `identity:get_user(user_id=e58593246fa743a79b081fce11a0140f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:17.483205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fced52e-dd76-4af7-b30f-6cbec58bf468 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:17.485312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 161/327] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:17 2022] GET /identity/v3/users/e58593246fa743a79b081fce11a0140f => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:17.491099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:17.557978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:17.600915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d644a3-c6eb-4f1d-9f43-52ab86ae1de0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:17.637737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:17.638904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.639354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.639824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] PATH_INFO: `/v3/system/users/e58593246fa743a79b081fce11a0140f/roles/0ae26fe95f5c41a8a3424f304b954f9d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.641541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e58593246fa743a79b081fce11a0140f, role_id=0ae26fe95f5c41a8a3424f304b954f9d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:17.643115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4fcbe-6408-4a31-b423-51ffac02ea17 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:17.657090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 167/328] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Jul 12 14:19:17 2022] PUT /identity/v3/system/users/e58593246fa743a79b081fce11a0140f/roles/0ae26fe95f5c41a8a3424f304b954f9d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:17.962786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7f6b8-95d8-4676-a7ac-944f344b38cb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.963082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7f6b8-95d8-4676-a7ac-944f344b38cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.963362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7f6b8-95d8-4676-a7ac-944f344b38cb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.965002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 162/329] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:17.970042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:17.970333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:17.970617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:17.991212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:17.994326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:18.079852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 168/330] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:18.121640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7f6b8-95d8-4676-a7ac-944f344b38cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:18.261326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7f6b8-95d8-4676-a7ac-944f344b38cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:18.304724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:18.305546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:18.305828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:18.306103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:18.324123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:18.332096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:18.332988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:18.333823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 163/331] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:18.343086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:18.413224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:18.465849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1620a-6450-4d66-9b3e-f52b70792cde None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:18.506191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:18.507410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:18.507970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:18.508944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:18.509926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:18.510427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:18.512168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:18.521700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 169/332] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:18.527474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:18.591059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:18.675077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48fe033-404f-496c-9351-eb30012ceb97 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:18.720655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:18.721609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:18.721941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:18.722223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] PATH_INFO: `/v3/users/02a36ca27cfb47398966b590635e8c57` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:18.723445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] RBAC: Authorizing `identity:get_user(user_id=02a36ca27cfb47398966b590635e8c57)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:18.724891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f734a-a754-44f8-8ab5-f67ec7093781 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:18.726927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 164/333] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:18 2022] GET /identity/v3/users/02a36ca27cfb47398966b590635e8c57 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:18.732981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:18.874361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720805d-0ba0-4022-a98a-4ad35368a140 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:18.896909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:18.922545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:18.923215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:18.923502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:18.923780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:18.924618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] RBAC: Adding query filter params (role.id=37d514b6250f41f1a7a945e02510a6df, user.id=02a36ca27cfb47398966b590635e8c57, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:18.924938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:18.926674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026023fa-2b92-4ffb-910b-e267ce6bec79 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:18.943338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 170/334] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 14:19:18 2022] GET /identity/v3/role_assignments?user.id=02a36ca27cfb47398966b590635e8c57&scope.system=all&role.id=37d514b6250f41f1a7a945e02510a6df => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:20.107269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f03f6-f3e2-4c75-979d-ab98007e1cd5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.107778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f03f6-f3e2-4c75-979d-ab98007e1cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.107852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f03f6-f3e2-4c75-979d-ab98007e1cd5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.109447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 165/335] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:20.115992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d31c1-a8e9-4fc0-8a9c-41b4c7934a1c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.116462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d31c1-a8e9-4fc0-8a9c-41b4c7934a1c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.116535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d31c1-a8e9-4fc0-8a9c-41b4c7934a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.141535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-be2d31c1-a8e9-4fc0-8a9c-41b4c7934a1c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:20.142745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-be2d31c1-a8e9-4fc0-8a9c-41b4c7934a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:20.233220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 171/336] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:20.269556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f03f6-f3e2-4c75-979d-ab98007e1cd5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:20.328579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f799418-8877-4186-8e78-975b3ec14398 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.328903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f799418-8877-4186-8e78-975b3ec14398 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.329148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f799418-8877-4186-8e78-975b3ec14398 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.330835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 172/337] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:20.335810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.336107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.336392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.382488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:20.384186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:20.436528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f03f6-f3e2-4c75-979d-ab98007e1cd5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:20.441685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:20.518482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:20.519282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.519564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.519840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.534510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 173/338] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:20.544394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:20.552359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:20.553228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:20.554525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 166/339] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:20.559523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:20.578556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:20.626120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:20.731288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a1323-0e1b-4563-b020-269adc733b90 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:20.767239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab793c-c85c-4c60-99e7-b569fc2f57c9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:20.771469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:20.772165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.772444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.772718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.774033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:20.774338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:20.775730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:20.799964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 174/340] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:20.806331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:20.818611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:20.819788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:20.820342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:20.820818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:20.843141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:20.855995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:20.856899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:20.857719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:20.864598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 167/341] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:20.865776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:20.901359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.001111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.004200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60777fc-fd06-4801-a095-7fbe0271b405 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.005770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09587bae-3aea-483c-9eda-f85ab61f8ea9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.045881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.046575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.046855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.047133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.047822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.047941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:21.048187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:21.049035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.049542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.049644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e51825-12c8-41bc-84b4-27e3390d12e2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:21.050100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.059133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 168/342] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:21.060469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f25a7e-d99e-4252-8465-266c1bde6424 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.060760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f25a7e-d99e-4252-8465-266c1bde6424 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.061042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f25a7e-d99e-4252-8465-266c1bde6424 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.066768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 169/343] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:21.071147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d204-e7d6-407d-8c2d-be4671399565 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.071703 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d204-e7d6-407d-8c2d-be4671399565 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.072280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7d204-e7d6-407d-8c2d-be4671399565 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.073694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 170/344] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:21.074334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7d204-e7d6-407d-8c2d-be4671399565 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.155838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:21.158081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45067dae-5940-4b3b-ae8f-5706c4a0ad8d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:21.194436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.219137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7d204-e7d6-407d-8c2d-be4671399565 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.267913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.267913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.268153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.268395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.287338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.293833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 175/345] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:21.298785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f90f-ea9a-476d-bc95-a16486e11bad None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.298785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f90f-ea9a-476d-bc95-a16486e11bad None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.298785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f90f-ea9a-476d-bc95-a16486e11bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.319849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7c59f90f-ea9a-476d-bc95-a16486e11bad None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:21.321628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7c59f90f-ea9a-476d-bc95-a16486e11bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:21.360034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:21.362064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:21.413359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 176/346] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:21.413359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3137-5e6a-4caa-bef8-e1bdc2777fa4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.413359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3137-5e6a-4caa-bef8-e1bdc2777fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.413359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b3137-5e6a-4caa-bef8-e1bdc2777fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.434766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3a6b3137-5e6a-4caa-bef8-e1bdc2777fa4 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:21.436510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3a6b3137-5e6a-4caa-bef8-e1bdc2777fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:21.475393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.497760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 171/347] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:21.498625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.525037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 177/348] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:21.527385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636e13-8211-4d43-a137-e5a684032f64 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.527669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636e13-8211-4d43-a137-e5a684032f64 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.527942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636e13-8211-4d43-a137-e5a684032f64 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.529367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 178/349] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:21.531059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd636e13-8211-4d43-a137-e5a684032f64 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.615413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.642443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b54a7-364b-4fbc-90cd-f2bf25415ac8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.674523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd636e13-8211-4d43-a137-e5a684032f64 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.690846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.692118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.692411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.692691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.693924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:21.695363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:21.697387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 172/350] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:21.698176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.717782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.719882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.720363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.720860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.737174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2075ad17-a74f-4145-bc9b-a4d617f3ff28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:21.743445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:21.744356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:21.745157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 179/351] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:21.746330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.746330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.751578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.774780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:21.776038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:21.776546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.839467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2707-3997-49e1-972c-4cc3f0a512c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:21.863966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 180/352] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:21.864805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.892734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:21.893672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:21.893953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:21.894225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:21.895414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:21.896852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:21.898646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 173/353] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:21.899407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:21.926973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:21.998024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d6e5b-459d-420c-a928-640d61d59a4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.035146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658096c-209d-4f78-84d8-0cbfe89b5a7c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.043627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.044451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.044735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.045090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.057437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:22.064330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:22.066892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:22.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 14:19:22.068981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:22.072140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 181/354] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:22.073366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.079882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.081236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.081586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.081861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.089450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.090917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.092621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 174/355] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:21 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.097471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.213194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaeff08-c019-4cf0-8acd-c0bc4053358f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.223860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.243062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743c7c5-f72f-48eb-85c5-9660bfa30c75 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.255662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.256342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.256645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.256901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.258236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:22.258543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.259923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.270097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 182/356] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.270749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.281972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.282838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.283123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.283404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.295318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:22.305326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:22.306216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:22.307005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 175/357] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:22.309177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.361782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.395124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53c127-3313-45d4-9785-f977a65f8a1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.431680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d18a2-b67e-4b98-baed-57595c9546e6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.437526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.438392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.438683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.438951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.440126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.441609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.443219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 183/358] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.443975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.475047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.475924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.476232 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.476515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.477270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:22.477622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.479118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57582c2-aaf3-4f93-950d-958023a3a060 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.488648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 176/359] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.489781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169101-8907-49d1-8116-ac902cb751b1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.490073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169101-8907-49d1-8116-ac902cb751b1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.490348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169101-8907-49d1-8116-ac902cb751b1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.491595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 177/360] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:22.492816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169101-8907-49d1-8116-ac902cb751b1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.496990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.568681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649bbb-00c2-4003-b0ca-e6675f40e862 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.607818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.608635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.608996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.609326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.610195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=9591568cf2904986aa8282568cb6b37b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:22.610512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.611949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.619399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 184/361] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=37d514b6250f41f1a7a945e02510a6df => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.620223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.626233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169101-8907-49d1-8116-ac902cb751b1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.640172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.678436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.679165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.680033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.680323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.766383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03dd7-c7e2-4bef-9ad5-6ae92867e841 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:22.768615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.774371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.777544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64ad8a-b353-4f9f-9fe6-806bd3705e9a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.818009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:22.820570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.820800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.821079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.822246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:22.822557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:22.823947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:22.834197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 185/362] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:22.834663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:22.908896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 178/363] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:22.910422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:22.910564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:22.910740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:22.928513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:22.929961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:22.932876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:22.971164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fda42-f126-476c-952d-25efcf25c7cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.010914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 179/364] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:23.017279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.018085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.018365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.018637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.019512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=9591568cf2904986aa8282568cb6b37b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:23.019880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.021322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.022157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.027841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 186/365] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:22 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.028647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.066557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.162604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7564c-5ffc-468b-bb04-743eb1717690 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.163207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592a05-d172-4e59-94ae-d8e6b97bb253 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.201140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.203408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.203918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.204406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.213873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.214759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.215043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.215322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] PATH_INFO: `/v3/users/e58593246fa743a79b081fce11a0140f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.216546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e58593246fa743a79b081fce11a0140f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.218172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.220274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 180/366] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/users/e58593246fa743a79b081fce11a0140f => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.221130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.237695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.293558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.296348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.335356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.342010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c302b97-5f74-41f5-bbae-1b01d89ed9f0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.387836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.388671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.388960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.389238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.390536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.392615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.394656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 181/367] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.395695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.451282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 187/368] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:23.452851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.504564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.544229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74157fb-52e4-4d93-b2a5-f44fa15c1426 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.587576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.588481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.588769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.589099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.603419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:23.612876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:23.615598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:23.616404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:23.618354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1aaed-433d-4ecc-9a30-bcb61243ebaf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.619053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 182/369] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:23.619926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.654876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.670034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.671039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.672099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.672564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.674104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] RBAC: Adding query filter params (role.id=0ae26fe95f5c41a8a3424f304b954f9d, user.id=e58593246fa743a79b081fce11a0140f, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:23.674610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.677180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.695328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 188/370] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/role_assignments?user.id=e58593246fa743a79b081fce11a0140f&scope.system=all&role.id=0ae26fe95f5c41a8a3424f304b954f9d => generated 510 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.697023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.739706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0f670-9914-49c3-9399-0eeaa16e96c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.789154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.791048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.791512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.792079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.794141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.796530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.799655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 183/371] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.800599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.809961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:23.831011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195375e2-fd13-4055-9cb9-c9e5ee4efeeb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.876154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:23.877373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:23.877511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:23.877842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:23.879291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:23.880916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:23.883431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 189/372] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:23.884308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:23.941098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a27-f48d-48ba-9cad-b70f737400f7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:23.944717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:24.001562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4321a9-20eb-4704-ba5e-98969a137236 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.011656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.013141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.013738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.014221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.015535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:24.016048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.017674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.030204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 184/373] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:24.037184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:24.051346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.052719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.053238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.053766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.055500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=f49c87f5ecd84589b554ef46f87ef734) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:24.056053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.057865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.065317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 190/374] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:23 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=37d514b6250f41f1a7a945e02510a6df => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:24.066748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:24.166881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:24.243440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864fe3d-0bd0-49c1-b58b-94cd33e8b3ef None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.271796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f343fc-7ccc-4f40-9151-6970e8e467dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.292820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:24.311835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.312473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.312757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.313032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.330106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.330978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.331261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.331535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.334115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.335836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.337702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 185/375] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:24 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:24.343630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:24.446511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.447980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.550558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73426ae-55e1-429c-980d-a426b6166c2f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.611027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:24.619711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.620375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.620662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.620942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.621837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=f49c87f5ecd84589b554ef46f87ef734) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:24.622155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.623597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.635887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 186/376] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:24 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 260 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:24.642384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 191/377] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:24.648680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:24.765741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293482b5-ec4c-4c35-9e18-f8c8d78a02b3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.803577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.804935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.805219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.805519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] PATH_INFO: `/v3/projects/63dcdffb6815414f8835106f58a82105` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.806793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] RBAC: Authorizing `identity:get_project(project_id=63dcdffb6815414f8835106f58a82105)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.808291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.810852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 187/378] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:24 2022] GET /identity/v3/projects/63dcdffb6815414f8835106f58a82105 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:24.822548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:24.847954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:24.949701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03217781-aa1f-4e38-956f-cf8bea62bb4f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:24.987152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:24.988028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:24.988311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:24.988589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] PATH_INFO: `/v3/groups/90ff069d77b94996b02e2f628d20fa2d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:24.995056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] RBAC: Authorizing `identity:get_group(group_id=90ff069d77b94996b02e2f628d20fa2d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:24.996513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7021ee-1ca4-4f0b-bc4d-1d3d2e3984e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:24.999533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 192/379] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:24 2022] GET /identity/v3/groups/90ff069d77b94996b02e2f628d20fa2d => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:25.005530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:25.089749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:25.141566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f525b-f310-4c97-8f0b-f5395502f6eb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:25.180033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:25.180697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.180977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.181248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.182116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] RBAC: Adding query filter params (group.id=90ff069d77b94996b02e2f628d20fa2d, role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=63dcdffb6815414f8835106f58a82105) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:25.182429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:25.183984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc629-c2ca-4a13-aea1-ad17783e2874 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:25.190195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 188/380] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/role_assignments?group.id=90ff069d77b94996b02e2f628d20fa2d&scope.project.id=63dcdffb6815414f8835106f58a82105&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:25.225599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd330-780b-43e1-ac88-23f6d7f34211 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.225969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd330-780b-43e1-ac88-23f6d7f34211 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.226182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd330-780b-43e1-ac88-23f6d7f34211 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.228426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 193/381] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:25.234451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.234737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.235003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.258456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:25.259675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:25.339977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 189/382] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:25.379936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd330-780b-43e1-ac88-23f6d7f34211 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:25.472081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:25.495109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd330-780b-43e1-ac88-23f6d7f34211 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:25.541054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:25.541879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.542162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.542440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.560370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0122ff73-7122-4949-99e5-7db32e7ca2f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:25.571243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:25.572091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:25.572863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 194/383] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:25.578692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:25.653219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:25.681896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2001e-3e8e-47c8-a2b5-9aaf398314ad None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.682227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2001e-3e8e-47c8-a2b5-9aaf398314ad None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.682459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2001e-3e8e-47c8-a2b5-9aaf398314ad None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.684072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 195/384] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:25.692039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.692328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.692604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.715621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:25.716823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:25.731941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaac59-9a92-48c0-8a19-1f39447ad685 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:25.787155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:25.787831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.788118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.788399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.789719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:25.790055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:25.792062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:25.801344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 190/385] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:25.807437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:25.922804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 196/386] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:25.925665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba5027-eac9-4c54-880f-bf9cd44789af None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:25.947334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:25.963073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:25.964139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:25.964541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:25.964920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:25.966245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:25.969833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:25.972442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:25.974535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 191/387] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:25.980602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.092545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:26.125710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6685a8e-d578-4bf7-93cc-76b26a168ec8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.161657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.162480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.162766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.163038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.174282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:26.183259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:26.184157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:26.184973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 197/388] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:26.192120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.211887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4c42-ebf7-4309-bdc2-3a775c980283 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.285287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.286715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.287218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.287689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.289837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:26.291555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:26.294129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 192/389] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:25 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:26.302571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.374419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aaf14f-aa0e-423f-bb27-72e6639cbb33 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.378621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:26.406786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:26.415637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.416411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.416762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.417174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.419065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:26.419479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:26.420952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589d360-07e8-4b24-a6ff-884c62feb03e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:26.432392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 198/390] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:26.433632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f089-d789-46f2-b01c-2b65f213d94a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.433990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f089-d789-46f2-b01c-2b65f213d94a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.434342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f089-d789-46f2-b01c-2b65f213d94a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.439818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 199/391] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:26.439818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f089-d789-46f2-b01c-2b65f213d94a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.456748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d046c-3f20-4e78-a86a-cc8774b03518 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.491814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.493013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.493518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.493976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/groups/b070a130f9324bd48d9e5bc3e835322c/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.496169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, group_id=b070a130f9324bd48d9e5bc3e835322c, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:26.497881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4029402-f0a0-478f-883b-112196c3df67 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:26.514034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 193/392] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 14:19:26 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/groups/b070a130f9324bd48d9e5bc3e835322c/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:26.515614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.516112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.516651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.548424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:26.550592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:26.591153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f089-d789-46f2-b01c-2b65f213d94a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.637550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 194/393] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:26.659346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.660285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.660617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.660898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.662107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:26.663693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:26.665300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.671025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 200/394] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:26 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:26.676647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.722746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:26.791179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132282d8-04bd-41f2-8484-8c1a07cf7e07 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.797695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:26.835165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.836620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.836893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.837174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.849614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:26.850125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8d52-c758-4d48-afe4-b72c893c1cb8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:26.859028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:26.859943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:26.860727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 195/395] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:26.866333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:26.907063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:26.908317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:26.908804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:26.909315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:26.911362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:26.914701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:26.917836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 201/396] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:26 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:26.924093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.087369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909c12-d12d-4974-984c-ccbf3d402416 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.096254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451284d-02a1-4f3a-bd61-f71054933f4c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.098570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.129360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.129360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.129360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.129360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.129773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:27.129773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.130880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.141202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 196/397] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:27.147398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.168546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.194060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.195917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.196409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.196891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b/groups/b070a130f9324bd48d9e5bc3e835322c/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.201833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] RBAC: Authorizing `identity:create_grant(project_id=9591568cf2904986aa8282568cb6b37b, group_id=b070a130f9324bd48d9e5bc3e835322c, role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.203565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.251955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 202/398] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 14:19:26 2022] PUT /identity/v3/projects/9591568cf2904986aa8282568cb6b37b/groups/b070a130f9324bd48d9e5bc3e835322c/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:27.262478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c7b2e-b60b-40fd-b9f4-be428291798b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.301787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.302724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.303024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.303347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.304761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.306416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f8984-0f91-4b44-b9c5-dd19fce9afeb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.308105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 197/399] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:27.314139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.346418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e2c8e-8bd0-460e-85a2-4c94d4366a4e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.346700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e2c8e-8bd0-460e-85a2-4c94d4366a4e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.346899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e2c8e-8bd0-460e-85a2-4c94d4366a4e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.348299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 198/400] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:27.354737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.355021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.355303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.386066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:27.387308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:27.454000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945aecc5-338e-40c1-b63a-395d5210d74e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.465413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.497688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.498832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.499119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.499541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.501370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.503320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.505716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 203/401] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:27.508011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 199/402] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:27.512701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.534709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.586621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.644177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f079a6-4d80-4448-98de-85f0c221313e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.660473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb833333-57cb-464e-8ed6-07a4909f71ba None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.685852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.686900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.687185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.687463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/groups/b070a130f9324bd48d9e5bc3e835322c/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.690237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, group_id=b070a130f9324bd48d9e5bc3e835322c, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.692296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20aaf8-0e11-454d-94b7-4c532b1aa2d7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.701469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.702902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.703686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.704171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.709046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 204/403] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 14:19:27 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/groups/b070a130f9324bd48d9e5bc3e835322c/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:19:27.711608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e742-04c9-490b-965c-d4fea5e93912 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.711907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e742-04c9-490b-965c-d4fea5e93912 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.712179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6e742-04c9-490b-965c-d4fea5e93912 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.713866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 205/404] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:27.720055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.720336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.720609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.724357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:27.733962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:27.734906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:27.735684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:27.738570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 200/405] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:27.739456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:27.740706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:27.744770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.777478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.828711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 206/406] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:27.886075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:27.900728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3befb99-21a7-4633-a433-1f849eaafd3f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:27.952202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:27.956789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:27.958883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:27.959449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:27.959933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:27.962002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:27.962578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:27.964340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:27.974203 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 201/407] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:27.980045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.011996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d87415-3a7b-4678-b02c-24ada7deff9e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.057718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.057718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.057718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.057718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.066581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:28.077910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:28.078801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:28.079579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:28.085532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 207/408] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:28.088112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.138919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:28.156072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f52c4-0006-40ac-afb3-edd9d1ae18d2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.221122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.222340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.222624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.222902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.225503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.226977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.232782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 202/409] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:27 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:28.233916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.272646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef83268-e57e-4ed9-929a-cc795443129c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.287867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:28.362098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.362775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.363058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.363323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.364071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:28.364375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.365778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.378034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 208/410] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:28.384458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.425454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18bed0-6ef9-48b6-aba3-32e559ff0324 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.491122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.492481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.492763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.493040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.498469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.500046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.501987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 203/411] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:28 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:28.522315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.537024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e0d66-2c54-4ec6-a744-38f24740c2a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.575153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.576391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.576894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.577401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] PATH_INFO: `/v3/groups/90ff069d77b94996b02e2f628d20fa2d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.579289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] RBAC: Authorizing `identity:get_group(group_id=90ff069d77b94996b02e2f628d20fa2d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.580945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.583242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 209/412] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:28 2022] GET /identity/v3/groups/90ff069d77b94996b02e2f628d20fa2d => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:28.588207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.605683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:28.632446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:28.703566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00280a08-62f1-4864-9a34-ebb32fc9d542 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.741002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.742292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.742654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.743006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] PATH_INFO: `/v3/projects/63dcdffb6815414f8835106f58a82105` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.744366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] RBAC: Authorizing `identity:get_project(project_id=63dcdffb6815414f8835106f58a82105)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.746587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.748733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 210/413] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:28 2022] GET /identity/v3/projects/63dcdffb6815414f8835106f58a82105 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:28.754486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:28.817042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b05c5-a57f-4d82-9179-fdbbe7a98244 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.881886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:28.883760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:28.884047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:28.884332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734/groups/b070a130f9324bd48d9e5bc3e835322c/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:28.888940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] RBAC: Authorizing `identity:create_grant(project_id=f49c87f5ecd84589b554ef46f87ef734, group_id=b070a130f9324bd48d9e5bc3e835322c, role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:28.890418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccff8e-3272-4c3e-ba83-1f1867208719 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:28.912442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 204/414] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 14:19:28 2022] PUT /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734/groups/b070a130f9324bd48d9e5bc3e835322c/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:28.947785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84761e32-975b-44f8-acd1-60e1ac49d0ec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:28.957189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:29.007618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:29.008522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.008806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.009083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] PATH_INFO: `/v3/projects/63dcdffb6815414f8835106f58a82105/groups/90ff069d77b94996b02e2f628d20fa2d/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.011108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] RBAC: Authorizing `identity:create_grant(project_id=63dcdffb6815414f8835106f58a82105, group_id=90ff069d77b94996b02e2f628d20fa2d, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:29.012562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abe094-fb6f-4b3e-9c46-d3acd603aa10 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:29.031144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 211/415] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Jul 12 14:19:28 2022] PUT /identity/v3/projects/63dcdffb6815414f8835106f58a82105/groups/90ff069d77b94996b02e2f628d20fa2d/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:29.159466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3dfc9-3d93-46fb-a449-1dbf73743d2e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.159820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3dfc9-3d93-46fb-a449-1dbf73743d2e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.160033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3dfc9-3d93-46fb-a449-1dbf73743d2e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.162200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 205/416] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:29.167951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.168250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.168541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.194306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:29.196113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:29.278699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 212/417] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:29.302424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3dfc9-3d93-46fb-a449-1dbf73743d2e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:29.429714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:29.446605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3dfc9-3d93-46fb-a449-1dbf73743d2e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:29.493905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:29.494713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.494999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.495275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.513731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:29.521702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:29.522573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:29.523952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 206/418] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:29.530688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:29.637674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:29.648916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffb26-5a7d-4138-84db-7cfb7ac89220 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:29.695736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:29.696416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.696695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.696969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.697752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:29.698057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:29.699707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcfb26-aa56-4d1e-9226-a33a485e789d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:29.708975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 213/419] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:29.714692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:29.859914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449cb0-7596-462b-a897-f065a62f83e2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.860243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449cb0-7596-462b-a897-f065a62f83e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.860467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32449cb0-7596-462b-a897-f065a62f83e2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.861861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 214/420] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:29.867787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.868307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.868783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.871132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691ced-28cf-47f1-badb-8b20ce09ce94 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:29.907052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:29.908614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:29.914216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:29.914929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:29.915220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:29.915502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:29.943858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.013194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 215/421] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:30.016897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:30.016897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:30.048938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.153428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 207/422] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:30.158323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.209713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.230362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c331fc-5e8a-4c58-822e-d4e5b65e93a0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.244047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.304991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.306339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.306826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.307303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.337183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:30.340710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2671fe-3487-46e1-ada4-5d574d916b6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4e82259b-b4f2-4ee4-b2f1-d675398f7bd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:30.345785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:30.346666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:30.348008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 216/423] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:30.348670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff559cb-67b6-483a-b35e-bb6ac3fa9ee0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.348670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff559cb-67b6-483a-b35e-bb6ac3fa9ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.348793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff559cb-67b6-483a-b35e-bb6ac3fa9ee0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.353803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 217/424] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:30.353803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff559cb-67b6-483a-b35e-bb6ac3fa9ee0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.392550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.393606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.393891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.394174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.395621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:30.397375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913880a5-2622-4026-ba01-ffac154db37d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:30.401056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 208/425] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:30.403015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.403304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.403641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.431248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:30.432591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:30.481932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff559cb-67b6-483a-b35e-bb6ac3fa9ee0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.515483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 209/426] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:30.516683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.535973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.536646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.536931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.537209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.537983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:30.538291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:30.542435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:30.555454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 218/427] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:30.556885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.573545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.579715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.651564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fafd72-26c1-44f7-950f-fde36d2f0106 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.691101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ae453-f3e4-47f9-8191-1258df37df64 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.701771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.703443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.704216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.704731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.706893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:30.709101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:30.711460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 210/428] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:30.713079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.728443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.729097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.729415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.729697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.740417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:30.748201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:30.749076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:30.749893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 219/429] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:30.750542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:30.829371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.863541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d405b-c7ee-4d26-b6e5-48ad3a7c7c76 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.864357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dc7c4-b055-4c47-be4f-0f0e3997f510 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:30.879786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:30.906449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.907403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.907541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.907849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.908698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=9591568cf2904986aa8282568cb6b37b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:30.909002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:30.910694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:30.913600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:30.915496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:30.916257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:30.916753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:30.918251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 220/430] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=37d514b6250f41f1a7a945e02510a6df => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:30.919214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.015785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.018082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.113295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1555c-b8b0-4b94-a5cf-15a049989e99 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.160292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.162016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.162532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.163046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.164588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:31.165028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 211/431] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:31.165403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.166920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.168495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.170781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.178303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 221/432] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:31.186146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.189196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.297252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85624-4e01-45b8-bd4e-c50036b107af None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.314498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb850-ada4-47a9-9e75-2ec70acdb50f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.340399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.341275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.341589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.341867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] PATH_INFO: `/v3/projects/9591568cf2904986aa8282568cb6b37b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.343071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] RBAC: Authorizing `identity:get_project(project_id=9591568cf2904986aa8282568cb6b37b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.344514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425470f-ae1b-4680-91c4-5d8c9bdd228f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.346547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 212/433] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/projects/9591568cf2904986aa8282568cb6b37b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:31.347739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8687dd-858c-426b-918d-78aa54581f7a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.348024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8687dd-858c-426b-918d-78aa54581f7a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.348300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8687dd-858c-426b-918d-78aa54581f7a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.349656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 213/434] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:31.356241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.356529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.356624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.356812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.357227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.357547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.357824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.373673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:31.374855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:31.445385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.454402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.455868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5a69f-1dff-495d-8801-7ae5cece4ba9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.474088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 214/435] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:31.475200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.488466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.574189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 222/436] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:19:31.575503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c976e9-1154-46a3-a811-86fa4e2b04c8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.575797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c976e9-1154-46a3-a811-86fa4e2b04c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.576074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c976e9-1154-46a3-a811-86fa4e2b04c8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.577511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 223/437] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:31.578113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c976e9-1154-46a3-a811-86fa4e2b04c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.596022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4a361-fb18-404a-aada-2281d17d9f47 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.630734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.631587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.631870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.632149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.633410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.634851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.636461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 215/438] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:31.637216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.672632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.680340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c976e9-1154-46a3-a811-86fa4e2b04c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.711674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.713718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.714813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.715068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.715346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.725186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e5f2e665-20ee-493e-8a1f-6d74b47cfc75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:31.733281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:31.734250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:31.735037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:31.736000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b860a3-06ab-4d37-9e74-8d5d2aea2f7d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.736071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 224/439] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:31.736071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.736071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.736242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.752390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:31.753613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:31.768950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.769824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.770109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.770385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.771550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.773518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.775489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 216/440] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:31.776244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.823916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 225/441] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:31.824799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.879257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6214b4e-00ae-4244-85f0-f77f02562702 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.908910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.915687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.916338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.916624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.916904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.917754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=9591568cf2904986aa8282568cb6b37b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:31.918064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.919478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:31.925300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 217/442] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=9591568cf2904986aa8282568cb6b37b&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:31.925863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd8585-ba90-4b6f-b0f9-10607bdd6d25 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:31.926240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:31.937543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:31.983181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:31.984351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:31.984643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:31.985075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:31.986017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:31.986394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:31.988324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.001521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 226/443] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:32.002435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.047775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c539274-8c37-4177-a803-c58fc781a29e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.090466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.091043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.091336 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.091615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.092825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.094300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.096087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 218/444] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:31 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:32.096808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.126815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f6416-7faf-4c92-a0f4-8b55287b8686 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.169197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.170017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.170302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.170578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.184085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:32.189282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:32.192239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:32.193093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:32.193944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:32.197589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 227/445] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:32.198362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.210277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.218848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8307d-d867-47ae-8ca9-e0cc658fe07b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.254413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.255068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.255356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.255711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.318164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842668a-9c1c-4e1d-ba7b-faa24ec39f7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.332895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.334380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.352755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.353616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.354041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.354423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.355552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=37d514b6250f41f1a7a945e02510a6df, scope.project.id=f49c87f5ecd84589b554ef46f87ef734) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:32.355868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.357737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.364145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 228/446] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=37d514b6250f41f1a7a945e02510a6df => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:32.365078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.419495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.445383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 219/447] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:32.451082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.459009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.496981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096aa91-f05f-43be-a847-cc3175858bb4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.535982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.535982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.536263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.536523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.537197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:32.537581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.539005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.549269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 229/448] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:32.555840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.567873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f3fa2-08f2-4723-befe-76dc9c24926a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.609378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.610192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.610483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.610766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] PATH_INFO: `/v3/projects/f49c87f5ecd84589b554ef46f87ef734` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.611969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] RBAC: Authorizing `identity:get_project(project_id=f49c87f5ecd84589b554ef46f87ef734)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.613429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.615137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 220/449] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/projects/f49c87f5ecd84589b554ef46f87ef734 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:32.622357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.664861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e66c1-9741-485e-a637-dc8bc03a1b4b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.681488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.713165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.713819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.714333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.714404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.717233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.744464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8a927-90c2-4244-9a0e-34fcbf260c59 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.779953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.780804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.781089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.781389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] PATH_INFO: `/v3/groups/b070a130f9324bd48d9e5bc3e835322c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.782555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] RBAC: Authorizing `identity:get_group(group_id=b070a130f9324bd48d9e5bc3e835322c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.784000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.785645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 221/450] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/groups/b070a130f9324bd48d9e5bc3e835322c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:32.790459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.791400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.792052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.888853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b1ab7-9947-4b6c-9a4b-98b661991d96 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:32.889023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 230/451] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 14:19:32.894296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:32.902396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.923158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:32.923803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:32.924084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:32.924357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:32.925173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] RBAC: Adding query filter params (group.id=b070a130f9324bd48d9e5bc3e835322c, role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=f49c87f5ecd84589b554ef46f87ef734) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:32.925512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:32.926811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:32.927178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:32.933292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 222/452] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/role_assignments?group.id=b070a130f9324bd48d9e5bc3e835322c&scope.project.id=f49c87f5ecd84589b554ef46f87ef734&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:32.995004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2c0fe-3d3d-4567-b5ef-fb4201f9b852 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:33.031252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:33.032131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:33.032414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:33.032692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] PATH_INFO: `/v3/projects/63dcdffb6815414f8835106f58a82105` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:33.033938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] RBAC: Authorizing `identity:get_project(project_id=63dcdffb6815414f8835106f58a82105)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:33.035400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:33.037137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 231/453] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:19:32 2022] GET /identity/v3/projects/63dcdffb6815414f8835106f58a82105 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:33.042817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:33.145617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f9c37-1388-4304-9385-2d59d9f842d8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:33.180013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:33.180867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:33.181152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:33.181461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] PATH_INFO: `/v3/groups/90ff069d77b94996b02e2f628d20fa2d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:33.182640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] RBAC: Authorizing `identity:get_group(group_id=90ff069d77b94996b02e2f628d20fa2d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:33.184102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94be2-43c8-4b25-9df9-be7886973f5a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:33.185888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 223/454] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Jul 12 14:19:33 2022] GET /identity/v3/groups/90ff069d77b94996b02e2f628d20fa2d => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:33.191659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:33.293423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0543ad-0a36-491c-8767-db58cf9155a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:33.294947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:33.327659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:33.328334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:33.328618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:33.328892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:33.329762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] RBAC: Adding query filter params (group.id=90ff069d77b94996b02e2f628d20fa2d, role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=63dcdffb6815414f8835106f58a82105) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:33.330060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:33.331490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47253d3f-e88d-4942-9ff1-b303fb11b63a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:33.337697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 232/455] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Jul 12 14:19:33 2022] GET /identity/v3/role_assignments?group.id=90ff069d77b94996b02e2f628d20fa2d&scope.project.id=63dcdffb6815414f8835106f58a82105&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:35.832409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869f43-31ca-4ef4-8c94-32c7c10d1da5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:35.832908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869f43-31ca-4ef4-8c94-32c7c10d1da5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:35.832908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5869f43-31ca-4ef4-8c94-32c7c10d1da5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:35.834515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 224/456] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:35.844594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2612-5352-4671-822a-371176e8ba72 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:35.845550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2612-5352-4671-822a-371176e8ba72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:35.846026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2612-5352-4671-822a-371176e8ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:35.882904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0db2612-5352-4671-822a-371176e8ba72 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:35.884460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0db2612-5352-4671-822a-371176e8ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:36.051876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 233/457] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:36.058840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90199e-df25-4d2c-9909-5acfe03198f5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.059401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90199e-df25-4d2c-9909-5acfe03198f5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.059874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90199e-df25-4d2c-9909-5acfe03198f5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.061449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 225/458] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:36.067111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b846b1e-3e40-4129-9bf2-309d8fc7871f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.067619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b846b1e-3e40-4129-9bf2-309d8fc7871f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.068106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b846b1e-3e40-4129-9bf2-309d8fc7871f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.094134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90199e-df25-4d2c-9909-5acfe03198f5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.105371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b846b1e-3e40-4129-9bf2-309d8fc7871f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:36.105371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b846b1e-3e40-4129-9bf2-309d8fc7871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:36.218317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.275053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 234/459] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:36.281202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436df5c-53ae-4055-9a66-5a95f7bd0c04 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.283683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436df5c-53ae-4055-9a66-5a95f7bd0c04 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.283683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436df5c-53ae-4055-9a66-5a95f7bd0c04 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.284804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 235/460] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:19:36.290750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.291033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.291312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.324266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:36.326533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:36.362922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.379822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90199e-df25-4d2c-9909-5acfe03198f5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.425372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.426543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.426930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.427387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.437643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 236/461] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:36.438790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.439754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.441751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:36.463278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:36.464176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:36.465023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:36.465023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:36.465023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.465023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:36.467158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 226/462] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:36.472747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.562363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76b7fb-194a-4eca-8067-3ea500f7edfe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.595465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.599836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.600636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.600954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.601236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.601991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948522a-803a-47ab-8117-09e5102e004e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.611396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.612857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:36.637657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:36.638568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:36.639445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:36.639445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:36.639445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.639445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:36.639647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.639647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.639647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.639851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.640410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 237/463] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:36.641262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.651474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.652769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.652946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-88927dfc-bc97-44d9-867c-ad89b0bc62cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:36.675154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:36.676005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 227/464] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c08e-0ce3-41c7-9bd3-2f8c125b7229 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c08e-0ce3-41c7-9bd3-2f8c125b7229 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.682314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c08e-0ce3-41c7-9bd3-2f8c125b7229 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.684637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 228/465] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:36.685612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66c08e-0ce3-41c7-9bd3-2f8c125b7229 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.750985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56d10-0349-4975-8b35-54b7a40bf770 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.787575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e5121-9093-4f47-987f-6835578326ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.788242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5121-9093-4f47-987f-6835578326ed None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.789050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5121-9093-4f47-987f-6835578326ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.789412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e5121-9093-4f47-987f-6835578326ed None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.790084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e5121-9093-4f47-987f-6835578326ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:36.790245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66c08e-0ce3-41c7-9bd3-2f8c125b7229 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.790401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e5121-9093-4f47-987f-6835578326ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.791777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e5121-9093-4f47-987f-6835578326ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:36.810524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 238/466] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:36.814914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e5121-9093-4f47-987f-6835578326ed None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.826564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.827247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.827835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.828117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.829111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:36.829453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.831516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ed41b-15d4-4778-b229-bab06e0017f1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:36.847429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 229/467] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:36.848778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.849074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.849383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.866405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:36.868205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:36.870457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.891657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:36.930765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e5121-9093-4f47-987f-6835578326ed None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:36.943184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 230/468] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:36.944175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:36.984951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:36.985930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:36.986705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:36.987136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:36.988321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:36.989082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:36.990880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.003905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 239/469] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:37.004877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.051522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b66468-5b59-45d7-bd93-c128cf7f2df5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.091185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.091862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.092147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.092425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.093129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.094633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.105619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:37.133360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 231/470] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:36 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:37.133360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.139050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1885e1-b825-4d65-af3b-d736b9d2f9ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.178024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:37.190688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.191491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.191930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.192282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.193383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.196076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.234482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 240/471] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:37 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:37.235318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.309126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865eeda-3a43-430c-b161-a8eefce78d64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.364202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.365057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.365360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.365705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.380717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef504-d636-4e59-863d-3744bf28b075 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.380914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.382847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8653ce36-2d95-48c6-b4dd-4ebfae205f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:37.410191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:37.411079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:37.411930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:37.411930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:37.411930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:37.411930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.411930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 232/472] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:37.412671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ee22-3e7e-45d9-aa5b-c1834a1e5ad6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.412671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ee22-3e7e-45d9-aa5b-c1834a1e5ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.412821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6ee22-3e7e-45d9-aa5b-c1834a1e5ad6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.415231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 233/473] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:37.422694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.423178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.423459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.435471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.436586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.437195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.437708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.438636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.440360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.440871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:37.442105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:37.471113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 241/474] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:37 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:37.472011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.514680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:37.537941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:37.563201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 234/475] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:37.590537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.624118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bd8f-4a22-4cc2-8146-bd8b19985182 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.673174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.673886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.674171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.674452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.675187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:37.675494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.677127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.694442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 242/476] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:37.699493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.706877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa6d2-24dc-45d4-8145-8f51edbb2e96 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.750161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.751080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.751517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.751960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.752853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.754416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.763297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.sql.core [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:19:37.763297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 14:19:37.763297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [parameters: {'id': 'd081842d17634fa698d140d4f41cc903', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 14:19:37.763297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80393) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 12 14:19:37.786669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:19:37.787659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application yield resource Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise self.value Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application session.commit() Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:19:37.788713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:19:37.790048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:19:37.791427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:19:37.792651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [parameters: {'id': 'd081842d17634fa698d140d4f41cc903', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:37.796033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 14:19:37.798174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:37.800969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 235/477] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 14:19:37 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:37.813460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:37.823443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:37.827645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7962b-6ef2-4a30-b57b-f422de3b8ee0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:37.869311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:37.870010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:37.870290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:37.870566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:37.871269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:37.873202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:37.900021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 243/478] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:37 2022] POST /identity/v3/users => generated 249 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:19:37.915842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:38.000195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4049a-a73d-449f-b6bc-9b618299ac6b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:38.047864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:38.049057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:38.049586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:38.050070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:38.068541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-dd5d2f94-7e88-478f-a227-7e8fbc8ce0c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:38.077770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:38.078709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 14:19:38.079508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:38.088391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 236/479] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:19:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:38.089063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:38.236321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d584b-46ac-44a2-891d-555d597c65c7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:38.271838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:38.272477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:38.272821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:38.273629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:38.274383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:38.274686 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:38.276065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba58cad-49ea-4821-af9b-fd7fbec40be4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:38.285033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 244/480] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:39.692454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065a26-7393-41cc-b8cd-2281ce730dd4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:39.692946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065a26-7393-41cc-b8cd-2281ce730dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:39.692946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065a26-7393-41cc-b8cd-2281ce730dd4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:39.695098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 237/481] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:39.700648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda0987-6acf-4964-ae70-eec37f4b2b9e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:39.701109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda0987-6acf-4964-ae70-eec37f4b2b9e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:39.701193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda0987-6acf-4964-ae70-eec37f4b2b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:39.726888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-efda0987-6acf-4964-ae70-eec37f4b2b9e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:39.728108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-efda0987-6acf-4964-ae70-eec37f4b2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:39.743500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:39.819005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 245/482] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:39.854555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87065a26-7393-41cc-b8cd-2281ce730dd4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:39.886405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1325-9cab-4318-9027-2bbccf960e91 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:39.886700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1325-9cab-4318-9027-2bbccf960e91 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:39.886972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f1325-9cab-4318-9027-2bbccf960e91 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:39.888549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 246/483] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:39.893945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:39.894231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:39.894510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:39.919154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:39.920349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:40.014014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.022365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87065a26-7393-41cc-b8cd-2281ce730dd4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.061354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 247/484] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:19:40.088673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.089878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.090012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.090494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.090974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.119623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:40.132271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:40.133269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.134103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:40.142082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 238/485] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:40.146203 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.208278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.233774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ad8ac-ae61-4663-a0f6-b6fe7817f2d0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.271396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.272703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.273160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.273604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.275661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a95f-262a-4021-834d-a217e56f8eed None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.284932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-904ef898-7080-4af0-94a2-ac1bd0f8eaaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:40.294260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:40.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.296236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:40.303443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 248/486] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:40.305229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f105d-c811-4f8e-9a0c-dd44a5efc679 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.305676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f105d-c811-4f8e-9a0c-dd44a5efc679 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.306117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f105d-c811-4f8e-9a0c-dd44a5efc679 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.308026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 249/487] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:40.308867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f105d-c811-4f8e-9a0c-dd44a5efc679 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.321726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.322765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.323046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.323319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.324170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:40.324794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.326493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e633d-bcaa-4380-a538-6e2393dfeefe None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.341552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 239/488] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:40.341552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.341552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.341552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.361685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:40.363343 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:40.382874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.439546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f105d-c811-4f8e-9a0c-dd44a5efc679 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.451917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 240/489] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:40.458534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.482984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.483666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.483946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.484221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.484966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:40.485272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.486679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.496020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.503744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 250/490] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:40.509053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.593645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a3fd4-912b-4112-9db6-a3a36e893c05 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.610020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0ba3-ccd9-4f35-baa8-67aca3825b38 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.633152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.634397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.634860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.635396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.645868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.646802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.647071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.647412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.648603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.650397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.658620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:40.666872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:40.667765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:40.668525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 251/491] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:40.669667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1c9afa2c-3fc9-476b-985d-0dee45e2d391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:40.671622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:40.672456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:40.673482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:40.673482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:40.673482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:40.673482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:40.673482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 241/492] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:40.675095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a73c-9395-4e71-ab48-783449f9564c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.675474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a73c-9395-4e71-ab48-783449f9564c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.675842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82a73c-9395-4e71-ab48-783449f9564c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.677483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 242/493] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:40.682659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82a73c-9395-4e71-ab48-783449f9564c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.687212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.737206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.781628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d89c3-56e5-4757-99a0-b79bebc586de None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.794291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82a73c-9395-4e71-ab48-783449f9564c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.814376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.815161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.815439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.815712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.827297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.828764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.828883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.829537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.829822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.830093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.830796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:40.831098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.832430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.843614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 243/494] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:40.844271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-305bb094-b895-4c97-8bf8-47fafa165608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:40.845790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:40.846793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:40.847749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:40.847749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:40.847749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:40.847749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:40.847749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 252/495] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:40.848368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ff839-29d8-4e13-8ab2-609097f664ed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.848664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ff839-29d8-4e13-8ab2-609097f664ed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.848938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ff839-29d8-4e13-8ab2-609097f664ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.865849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-889ff839-29d8-4e13-8ab2-609097f664ed None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:40.867554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-889ff839-29d8-4e13-8ab2-609097f664ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:40.908511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.933307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 253/496] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:40.938304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aac8-27b6-4e31-943f-a1247f832e2f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.938598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aac8-27b6-4e31-943f-a1247f832e2f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.938881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aac8-27b6-4e31-943f-a1247f832e2f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.940159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 254/497] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:40.940849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022aac8-27b6-4e31-943f-a1247f832e2f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:40.949414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743717f3-7ca2-4668-92f8-88a2a550429e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:40.985089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:40.985778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:40.986064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:40.986342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:40.986636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:40.987094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:40.987406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:40.988796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:40.997567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 244/498] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:40.998377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.044094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022aac8-27b6-4e31-943f-a1247f832e2f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.077649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.078434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.078713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.078989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.085903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.087341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.097640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50d0c-01a6-4893-aef4-1fab406bb33b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6ca58265-01cd-47c7-a2f5-5e1d84b135ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:41.103076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:41.104087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:41.105199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 255/499] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:41.106090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.106090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.106090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.124166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:41.125394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:41.125782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.135202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.135851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.136128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.136400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.137116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:41.137443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.138791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.150358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 245/500] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:41.151185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.191859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 256/501] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:41.192682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.225610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.263763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9460-bc8f-4db7-8cd3-19f510ef50c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.294792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f4207-ab27-45dc-9625-3ee72f3e4e11 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.297355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.298149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.298437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.298719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.309105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:41.316860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:41.317759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:41.318520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 246/502] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:41.319154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.333477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.334297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.334575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.334846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.345266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.346769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:41.365111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:41.366168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:41.367628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:41.367628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:41.367628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:41.367628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:41.367910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.367983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 257/503] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:41.367983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.423001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89ae8-b5f0-4252-b910-02d394e9e8f4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.452590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.456972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.457666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.457947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.458217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.459035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:41.459340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.461277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.468938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf4232-b860-4636-be7b-9acefc7377b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.497641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 247/504] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:41.498467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.511427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.512226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.512510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.512789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.520114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.521599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:41.536348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:41.537377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:41.538283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 258/505] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:41.539417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.598060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de9a24-d36a-4951-a5fb-f1795c696c4d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.600370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.632400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.633118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.633433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.633717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.637458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f86be-2863-4303-9ff1-08ff2a3e59d9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.643672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.645128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:41.662707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:41.663581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:41.664392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:41.664392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:41.664392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:41.664392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:41.664392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 248/506] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:41.664769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.674605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.675274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.675559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.675833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.676838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:41.677148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.678553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.687305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 259/507] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:41.688279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.698020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.763907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b1ee9-990f-49de-9ce1-edbd1f94b9fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.785906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5890a51-85c0-4721-829f-1d5b8f104d88 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.798057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.798712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.798990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.799263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.799977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:41.800282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.801951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.813792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 249/508] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:41.814573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:41.823599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.824247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.824838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.824838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.831632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.892019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.893431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.901648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:41.911745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee002d6-b301-42e6-950a-594f87e18cf3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:41.945823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:41.946484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:41.946768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:41.947043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:41.947876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:41.948188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:41.949636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:41.967506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 250/509] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:41.968307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.002685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 260/510] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:42.003523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.042091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.077186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93b398-c34d-4b8e-991e-121d7fb2e8ec None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.106618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524a696-b521-4e8d-9d3d-0f5d3bf7d336 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.111672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.112326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.112607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.112883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.113846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:42.113964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.114037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.115483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.128077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 251/511] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:42.128871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.142760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.143535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.143818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.144135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.154918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.156371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:42.173547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:42.174439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:42.175506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:42.175506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:42.175506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:42.175506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:42.175506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 261/512] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:42.176482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.234126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1884f-db29-41e0-84ed-44ae38b60dcc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.267754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.268540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.268819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.269096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.271047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.277872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.279312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.286619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff8e0-b108-4367-be93-808f0620bddf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:42.294652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:42.295647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:42.296532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 252/513] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:42.297108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.323009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.323730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.324007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.324280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.352212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.394520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.395910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.406566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000abf0-bcbd-4c35-b25c-16369bb4340b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.441472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.442289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.442571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.442846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.453966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.455413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:42.472141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:42.473009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:42.473880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:42.473880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:42.473880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:42.473880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:42.473880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 253/514] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:42.474157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.501707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 262/515] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:42.502720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.522364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.559899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.577772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b9bf8-828f-4174-ad2c-291688ff3add None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.601587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2f45a-e310-4e9d-954b-b63d3a686ff6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.612105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.612777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.613060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.613374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.614076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.615510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.635691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 254/516] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:19:42 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:19:42.636502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.636606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.637387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.637673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.637949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.638678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:42.638979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.640401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.655334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 263/517] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:42.655334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.760844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2341ed2-db1f-41d9-abd0-f9d534bc5860 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.762521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.778319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7b1b7-5797-46e5-b63c-e0b60cf7ac01 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.797575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:42.805861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.807070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.807536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.807817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.808666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:42.808985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.810454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.816329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.816997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.817282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.817589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.818313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:42.818622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.820523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.831612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 255/518] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:42.832864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.836302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 264/519] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:42.836302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:42.938383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557a51-26bb-490f-9d16-4e2aa0479eb9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.949840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffd1a-f615-43cc-a819-b729be10124c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:42.974922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.975983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.976391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.976673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.984533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:42.984838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.985324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:42.985639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:42.985909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:42.986454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:42.996983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:42.998451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:43.003345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:43.004194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 265/520] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:43.010292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:43.023357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:43.024202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:43.025020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:43.025020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:43.025020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:43.025020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:43.025020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 256/521] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:43.025770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.045177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.051861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.114898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b79727-ce16-4c2e-a631-2fe03e9bd771 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.143521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca74231-2fd7-475e-842c-c4e8075a9ab1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.156294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.156950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.157235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.157598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.185851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.186645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.186926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.187192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.194205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.196147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:43.211971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:43.212873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 257/522] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:43.219086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.236074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.237857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.278117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.295639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.331195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cce2e-65fa-4872-9b8a-65c782b01a34 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.345371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 266/523] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:43.345855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.372492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.373168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.373494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.373774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.374642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:43.375077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.376368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.398922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 258/524] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:43.398922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.462818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df0343-4cd6-43d7-95ab-38624dff40c2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.496885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.497573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.497858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.498133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.498847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:43.499155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.500579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.506869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.519376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 267/525] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:43.519376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.526835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd2784-2e26-42cf-bed1-2226a1c8575b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.559474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.566396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.567149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.567439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.567716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.568503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:43.568817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.570363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.588662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 259/526] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:43.589492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.632482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737546e-3ebc-49d2-b847-26fd6105e100 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.668007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.668799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.669081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.669377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.681003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.682507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.695285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eceb48-1a3c-4bc1-919c-8e1dbee1d41c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:43.700402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:43.701235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:43.702104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:43.702104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:43.702104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:43.702104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:43.705777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 268/527] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:43.707152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.731101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.731972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.732399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.732817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.762802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.794308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:43.808383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.809889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.830943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e274a2-f7b0-42d2-ab65-eb7bdc71b534 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:43.866929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:43.867841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:43.868306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:43.868459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:43.875826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:43.877288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:43.892318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:43.893233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 269/528] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:43.899354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:43.926085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 260/529] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:43.926085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.012879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae65756-99f1-48f5-95db-eb0de7fd303d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.017136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.025236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.026989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a435c84-48f6-46fc-972c-aaa03196a9ce None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.056808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.057496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.057781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.058058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.058888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.059197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.060658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.062702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.063641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.063641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.063885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.064596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.064904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.066293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.077676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 270/530] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.077676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.096063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 261/531] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.097358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.197109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fa0dc-042f-48fe-8e32-de34e752ba9e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.216814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2be42-4f82-4e76-af22-bb352cf1c070 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.236053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.236822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.237104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.237431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.238231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.238544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.240001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.255607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.256404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.256692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.257536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.259108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 271/532] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.259956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.269973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.271492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:44.290717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:44.291612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:44.292992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:44.292992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:44.292992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:44.292992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:44.296876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.297749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 262/533] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:44.298946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.303775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.368483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287e29-0298-426b-8a42-4fc0fba5ca49 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.422880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.423688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.423967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.424246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.434429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.435905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.440661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a0164-d64d-4221-8556-a1892cd2465b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:44.450038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:44.450895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:44.451759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 272/534] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:44.452678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.492068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.493100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.493630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.494112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.495146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=0df73889cb3f4bb58bdb5d674ed5379d) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.495665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.497883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.507174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 263/535] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/role_assignments?user.id=0df73889cb3f4bb58bdb5d674ed5379d&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.508227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.591342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.619246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.637356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d934c-bd52-43ec-8ca1-24e06c03bb47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.640001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e04840-98d6-4a1a-a1f8-742fc4164a9b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.683189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.684757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.685277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.685788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.687117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.687884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.689587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.705782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 264/536] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.711858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.719523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.722134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.727287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.727870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.728865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.729419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.731056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06454c5f-5cbe-4ff4-8975-ff4f2345c56e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.756491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 273/537] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.758757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222a9-39ab-41c4-84d0-e4b4d4a72968 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.761112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222a9-39ab-41c4-84d0-e4b4d4a72968 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.761112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222a9-39ab-41c4-84d0-e4b4d4a72968 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.762031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 274/538] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:44.767960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.768249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.768532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.788175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:44.790679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:44.864768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 275/539] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:44.866480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.875901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e07f2-f351-49ff-a293-01717af6cd96 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:44.918445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:44.928567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:44.929152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:44.929483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:44.929757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:44.930595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=50b29bba6ee54cab9e3f6fdcc841ec77) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:44.930912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:44.932337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:44.938705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 265/540] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/role_assignments?user.id=50b29bba6ee54cab9e3f6fdcc841ec77&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:44.939598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:44.997110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:45.029803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368d46a-0334-4a76-9e11-26dd522ce984 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.059073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893d01d-73f1-4fe9-ac9f-8fade5fcccac None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.096013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.097153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.097694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.098174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.106098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.107148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.107256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.107453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.111801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:45.118832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.120543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:45.122763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:45.124060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:45.125235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:45.127800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 266/541] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:45.134147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:45.158400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:45.159490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:45.160418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:45.166142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 276/542] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:45.178541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.197236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:45.244689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2801e-bdb6-4260-8e9f-6fd2ac6a9046 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.280807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.281847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.282326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.282793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.283749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:45.284262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.285837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.298236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 267/543] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:45.315158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.438842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a824e69-3058-4019-9bd7-d30811e6bc4c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.496589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.497242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.497545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.497812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.498588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:45.498892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.500311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.504102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc5ae-d9e8-41ce-96c1-49d43223fd7d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.524944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 277/544] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:45.526980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:45.533315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.545672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:45.573646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.574434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.574716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.574988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.597978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.599480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:45.628280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:45.629141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:45.630012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:45.630012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:45.630012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:45.630012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:45.638318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 268/545] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:45.646791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.669735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b1c8e-3f30-423f-aa82-8c3aaf7f672e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.730163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.731078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.731511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.731961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.732951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=2b899731a1ef4d9b961492dd78632ac5) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:45.733446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.735038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.749910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 278/546] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/role_assignments?user.id=2b899731a1ef4d9b961492dd78632ac5&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:45.785273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a4b0-3bf2-4788-8e08-510addccacfe None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:45.826282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be430-8254-4204-8403-f4058a925abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:45.826958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be430-8254-4204-8403-f4058a925abb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:45.827238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be430-8254-4204-8403-f4058a925abb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:45.827515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be430-8254-4204-8403-f4058a925abb None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:45.828243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be430-8254-4204-8403-f4058a925abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:45.828543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be430-8254-4204-8403-f4058a925abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:45.830105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be430-8254-4204-8403-f4058a925abb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:45.849447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 269/547] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:45.854449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:45.973003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.011287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120ba9e-83e4-4063-a371-147360376eb1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:46.045503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:46.046352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.046630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.046947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.062396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:46.063963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:46.081705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:46.082654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:46.084040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 279/548] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:46.094463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be430-8254-4204-8403-f4058a925abb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:46.131812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.227694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be430-8254-4204-8403-f4058a925abb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:46.273039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:46.273702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.273981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.274258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.275150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:46.275458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:46.276928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4fba7-b915-45c1-8a27-2f27d77f6da5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:46.302402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 270/549] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:19:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:46.308619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:46.441811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edd90-f6fb-4f33-86fe-5ead2c671153 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:46.484540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.492918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:46.494663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00f500-5e3e-4def-b1d1-748e1e68cae0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.494946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00f500-5e3e-4def-b1d1-748e1e68cae0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.495217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00f500-5e3e-4def-b1d1-748e1e68cae0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.496575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 271/550] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:46.497123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.497434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.497706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.506506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.506793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.507106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.529407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:46.530765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:46.605224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 272/551] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:46.638239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:46.638644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:46.639707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:46.664197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.789881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 280/552] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:46.802484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46997a43-47ec-4451-bcd1-16cb0bf388fd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:46.806068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:46.845252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.853585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:46.854536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.854821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.855098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.870943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-dc4c8e7e-c552-4a96-a5d2-73f0cd5b6d3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:46.879179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:46.880176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:46.880985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 273/553] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:46.882706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84822978-9c3d-4099-853a-b3fe5df17d0a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.882918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84822978-9c3d-4099-853a-b3fe5df17d0a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.885363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84822978-9c3d-4099-853a-b3fe5df17d0a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.888361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 274/554] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:46.889150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84822978-9c3d-4099-853a-b3fe5df17d0a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:46.929706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a919d5-78cc-4f2b-a0eb-98110d3d7080 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:46.964289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:46.971020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:46.972277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:46.972576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:46.972854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:46.986190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:46.988354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.002150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84822978-9c3d-4099-853a-b3fe5df17d0a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b25f4d0f-4343-42c6-9f50-d16115309ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:47.010343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.012419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.014512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:47.014512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:47.014512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.014512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:47.021356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 281/555] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:47.021356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.021356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.021356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.039667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:47.041510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:47.050533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.051654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.052475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.052880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.054679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:47.055381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.057145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.073586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 275/556] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.074564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.125602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 282/557] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:47.126584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.139771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.212645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79950892-e51f-4662-b5e9-115fa03c8aca None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.236096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d6513-8f40-4a56-85dc-c6b763d95421 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.252824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.253610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.253957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.254302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.255097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:47.255578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.257174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.258767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.271652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 276/558] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.273162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.277616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.278593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.279013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.279423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.290916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.292994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:47.314318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.317200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:47.317200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:47.317200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.317200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:47.323836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 283/559] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:47.325130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.383066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085b200-e9a6-4e92-9748-1fa1c2050b6b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.392660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.429640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.430333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.430697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.430931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.438622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95fc8a-3d33-4a0a-aa94-8f9f7b2def15 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.442478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:47.450799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.451705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:47.452607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 277/560] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:47.453275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.483636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.484467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.484753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.485070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.493294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.494765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6d5c980f-a1c6-4d63-8d17-4b64e827c32e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:47.508516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.509403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:47.510325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:47.514204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.514869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 284/561] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:47.516280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe774-b64a-44e1-8949-26a991a5305d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.516645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe774-b64a-44e1-8949-26a991a5305d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.517020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfe774-b64a-44e1-8949-26a991a5305d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.522547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 285/562] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:47.522547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfe774-b64a-44e1-8949-26a991a5305d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.569043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b3981-006b-499a-adab-00a25ef4be68 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.604361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.605247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.605623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.605901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.606625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:47.606934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.608296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.625178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 278/563] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.625178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.635411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfe774-b64a-44e1-8949-26a991a5305d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.658648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.671453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.672122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.672405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.672680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.673455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:47.673758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.675136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823656-52c1-4d75-a48e-4db4cf49e3c7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.688395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 286/564] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.688395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.688395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.688395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.705564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:47.706757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:47.731480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac5d03-2897-462d-8d30-04410b82ca47 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.746922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.770295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.770952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.771237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.771518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.772302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:47.772642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.774100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.776925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 287/565] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:47.777815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.798458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 279/566] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:47.799698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.881321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a4d58-7c35-4f6f-a369-8ef9e416a894 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.897079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:47.914679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd79e-94ea-4bd8-a44c-855517655b73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:47.920617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.921440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.921725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.922008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.929554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.931002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:47.945601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.946609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 288/567] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:47.947546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:47.956400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:47.957198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:47.957516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:47.957792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:47.969998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:47.971452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:47.989614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:47.990500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:47.992041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:47.992041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:47.992041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:47.992041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:47.993565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 280/568] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:47.994487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.028684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.066101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec9af8-7208-4753-b7ab-8fb0b11ff983 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.102274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.103077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.103385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.103662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.114077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6c78d-bf24-47c7-ac5e-115f420a778b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.114687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:48.123367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:48.124504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:48.125535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:48.131444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 289/569] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:48.132695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.156101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.156847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.157194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.157513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.158339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=9f47a2120e91452ebcaa6e042caed9ed) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:48.158653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.160095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.165867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 281/570] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:47 2022] GET /identity/v3/role_assignments?user.id=9f47a2120e91452ebcaa6e042caed9ed&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:48.166646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.173987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.271692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda81f1-f8a1-4532-bd30-ac67e4cd9a6a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.280494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.295005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd858-826d-496d-ac07-0c9010c2d8a0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.314120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.314790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.315177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.315524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.316445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:48.317293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.318943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0744d8e-2fde-4b62-8381-870c4e52a896 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.333666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.334552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.335043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.335991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.336864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:48.337477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.339103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.356921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 282/571] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:48.358081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.358336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 290/572] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:48.359652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcabcf5-9d6c-4317-b3b7-67d31f98dcb7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.359943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcabcf5-9d6c-4317-b3b7-67d31f98dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.360303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcabcf5-9d6c-4317-b3b7-67d31f98dcb7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.361724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 291/573] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:48.362250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcabcf5-9d6c-4317-b3b7-67d31f98dcb7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.468069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.478624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39727-5b7b-46b3-ad90-5a43970fe8d6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.479098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcabcf5-9d6c-4317-b3b7-67d31f98dcb7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.515613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.515823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.516277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.516447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.516543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.516728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.516825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.517005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.517593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:48.517949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.519267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f52260-aac7-4df3-b847-965912dcd13f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.525218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.526689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.533096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 283/574] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:48.533096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.533096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.533096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:48.547025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:48.547966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:48.554361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:48.554899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:48.554899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:48.555027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 292/575] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:48.555027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.561898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.628067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 284/576] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:48.628903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.671480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd118-fc62-40f2-9629-51e64151feba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.706884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.707681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.707954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.708220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.719340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.720838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.738207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270380f-d516-4e57-a2de-06bce51227e7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:48.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:48.740208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:48.741081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:48.741081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:48.741081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:48.741081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:48.741081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 293/577] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:48.741917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.747563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.778030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.778811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.779092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.779370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.789742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.791185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.793496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:48.810885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:48.811826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:48.812708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:48.812708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:48.812708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:48.812708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:48.812708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 285/578] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:48.813417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.852479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f350dc-a3e6-4adb-b943-96ce5b34cc19 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.889078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.890176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.890176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.890436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.891215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:48.891528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:48.892966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:48.912099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 294/579] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:48.912982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:48.926500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b79a06-918b-483f-a0b1-ff9089a28919 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:48.961278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:48.962173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:48.962463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:48.962732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:48.974038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:48.977838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:48.982566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:48.983487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 286/580] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:48.985010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.023505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60309-6ca9-434a-a980-6ba6fa7565be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.043529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.058993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.059650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.059923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.060198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.060914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.061217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.062603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.074547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 295/581] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.075348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.100414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b6344-215e-4c68-8b61-a584657b62fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.137058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.137984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.138270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.138786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.139904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.140342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.142133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.160926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 287/582] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.160926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.187833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5b024-9565-4e0e-990d-75eef112bad7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.213479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.228483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.229282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.229615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.230040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.243578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.245640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:49.263616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:49.264593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:49.265507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:49.265507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:49.265507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:49.265507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:49.265507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 296/583] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:49.265811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.274366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e001be-f4e5-4c42-ae2f-c5f3e4caed75 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.289222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.311218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.311880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.312326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.312444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.313147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.313481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.314828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.323067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 288/584] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.323843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.379033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70bc0f-62e0-4cbc-b95e-6964f9212432 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.419747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.420553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.420830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.421099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.427783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530523-9a3e-4c71-a0e4-439f807c32f8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.428151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.430126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:49.444189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:49.445126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:49.446060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 297/585] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:49.446572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.465659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.466440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.466722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.467002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.474989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.476497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.485511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:49.490905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:49.492405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:49.493324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 289/586] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:49.493950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.530701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.561651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba1e1c-def8-4493-9b38-72c88e23768a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.597134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68f207-4c5b-4715-ada3-382c031d950b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.599390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.600063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.600345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.601161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.601917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.602226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.603658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.615786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 298/587] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.616669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.636435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.637212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.637525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.637806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.649839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.651286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:49.670046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:49.670969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:49.671841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:49.671841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:49.671841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:49.671841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:49.675822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 290/588] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:49.676751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.718633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.727230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e8e3-f5b2-4b14-b2f0-687f17810d1c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.765985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.766970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.767431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.767886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.768846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.769451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.771071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.771812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:49.789362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 299/589] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.789362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.789606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d75ae-5819-4578-8d72-51a20bf3a9c1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.825452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.826134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.826416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.826690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.827529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:49.827840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.829267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.847558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 291/590] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:49.848377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.896362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739ddb9-b219-4f65-b374-afa52b9fb594 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.931319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:49.932106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:49.932390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:49.932662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:49.940659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:49.942129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:49.956781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:49.957752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:49.959282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:49.959723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d7a74-da4a-42ac-bcc7-dd0681ea845b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:49.959799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 300/591] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:49.959799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:49.978050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.000065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.000723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.001002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.001275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.002009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:50.002314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.003658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.019518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:50.019518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.026148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.077835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54333-2951-46c7-a7c6-172837c48bf9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.117676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.118748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.118859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.119271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.126940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf75d35-4f8f-471c-834b-9fb800a7f5c4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.136249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=a1fb1196840240b4aa05a2f83d4c7ee7, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.137701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.152119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 301/593] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:49 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:50.153205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.178363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.180343 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.180631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.180911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.197062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.198952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:50.219931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:50.220915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:50.221841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:50.221841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:50.221841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:50.221841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:50.221841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 293/594] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:50.222121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.244520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.285620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.292810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c9811-4596-4229-96da-4e5445629ab8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.337227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.337655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8456bf1-6335-4526-a104-cb98c4b1c5e1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.341875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.342165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.342445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.343245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:50.343558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.345147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305461e0-8601-4332-8032-69d7785ccd2f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.358684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 302/595] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:50.360007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921744-465b-4795-8570-2186cee8d82e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.360255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921744-465b-4795-8570-2186cee8d82e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.360531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921744-465b-4795-8570-2186cee8d82e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.362095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 303/596] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:50.362801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51921744-465b-4795-8570-2186cee8d82e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.376926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.378014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.378475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.378956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.387516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.389188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:50.404722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:50.406118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:50.407365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:50.409481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 294/597] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:50.413736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.484201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51921744-465b-4795-8570-2186cee8d82e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.509941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.520560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb2bf-4b4c-4446-bc84-ba8ed3d5bc28 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.522865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.523597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.523890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.524177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.524904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:50.525213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.526627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1c6b-3e69-49b5-b97f-fc9d30cebb4d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.539245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 304/598] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:50.540538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.540836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.541118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.544485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.560281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:50.561499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:50.562331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.563222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.563521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.563772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/0df73889cb3f4bb58bdb5d674ed5379d/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.581850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=0df73889cb3f4bb58bdb5d674ed5379d, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.584258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.598636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 295/599] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:50 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/0df73889cb3f4bb58bdb5d674ed5379d/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:50.599711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.651019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 305/600] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:50.652243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.745603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e5f90-3808-4bfc-af56-a2057935fb95 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.772262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c031d3-99a8-4571-96a3-25ce6ef992cf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.784007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.784846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.784958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.785243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.786125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:50.786438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.787869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.793021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.810576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 296/601] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:50.815073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.832770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.836222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.836623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.836994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.844731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:50.845435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:50.846937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:50.864137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:50.865293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:50.866431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:50.868375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 306/602] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:50.869472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:50.947191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac15d59-ea97-4bf3-b73e-36467e2f1d60 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.982591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75969c56-ce2d-4a1f-bee9-641fb95ba146 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:50.983621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:50.984591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:50.984904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:50.985274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:50.996521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:51.004459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:51.005625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:51.006653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:51.008783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 297/603] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:51.009923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.028117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.029139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.029727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.030179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.043254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.044840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:51.064141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:51.065270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:51.066412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:51.066412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:51.066412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:51.066412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:51.068342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 307/604] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:51.069420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.104339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.142732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32dd36-6c0f-44ef-a1f6-7ddadf521dab None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.184093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f305b-c85b-4764-9152-f24bf1a3a7e4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.186230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.187001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.188273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.188662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.189591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:51.190029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.191567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.209667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.212191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 298/605] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:51.213038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.227514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.228585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.229125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.229719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.230719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:51.231231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.232898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.245379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 308/606] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:51.245379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.330423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4821-2803-42cb-b7b5-675e6ec1976f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.342275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.356284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10612233-ff14-457c-ac11-6fd5ef2486b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.368079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.368701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.368902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.369304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.370076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:51.370363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.371739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.388549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 299/607] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:51.389540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.395484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.396651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.397143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.397730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/50b29bba6ee54cab9e3f6fdcc841ec77/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.399992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.414614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=50b29bba6ee54cab9e3f6fdcc841ec77, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.416535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.441374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 309/608] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:51 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/50b29bba6ee54cab9e3f6fdcc841ec77/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:51.442787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.524092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2eaa84-bc1b-45a0-b9b6-7c1c6d0e29bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.566008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.566954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.567332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.567752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.581914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.583851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:51.605278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:51.606422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:51.607596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:51.607596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:51.607596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:51.607596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:51.609061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 300/609] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:51.615886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.689815 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.694599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b2c1-5b3a-46f9-a3a1-8b55fc38de7b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.740535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.740999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.741996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.742478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f37d-1a4b-4b4b-8514-daeff8b3b936 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:51.750651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.753122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.757011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:51.778775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:51.779725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:51.780614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:51.782723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 310/610] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:51.788866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.803454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:51.804880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:51.805359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:51.805787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:51.806675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:51.807139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:51.808664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:51.831642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 301/611] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:51.837672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:51.977839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b85de-947b-4f7b-b3df-4d5617404b47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.009898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48323b-3946-42ce-861e-682b527d923c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.029768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.050212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.050882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.051167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.051699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.052496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:52.052812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.054342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.066024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.066831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.067112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.067356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 311/612] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:52.067467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.074098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.075656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.077102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:52.115162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:52.116169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:52.117918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 302/613] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:52.126363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.139050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.210087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8829ef-a1be-42aa-a0ab-a6992cdce37a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.265412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.266313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.266601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.266878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/2b899731a1ef4d9b961492dd78632ac5/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.285165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=2b899731a1ef4d9b961492dd78632ac5, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.287070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e5384-b6fa-4559-b245-c3f4a8097eb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.302958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 312/614] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:52 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/2b899731a1ef4d9b961492dd78632ac5/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:52.337020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3b12-ee64-46ab-a1f3-90ca1c5b5772 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.352065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ec-e0fc-4dc0-ada9-3b54f5719693 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.352401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ec-e0fc-4dc0-ada9-3b54f5719693 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.352635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f174ec-e0fc-4dc0-ada9-3b54f5719693 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 313/615] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:52.364197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.364521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.365300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.400403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:52.401652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:52.405372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.406121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.406408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.406687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.407538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:52.407851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.409303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.439662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 303/616] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:52.444099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.483020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.499498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 314/617] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:52.529044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.565314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eec12-5620-421b-b64d-8dbce81a7ad0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.576714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.617067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.617821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.618109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.618419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.633695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.635409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.653583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5843e-d91b-4837-81d0-237d02ab5396 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:52.665410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:52.666310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 304/618] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:52.667199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.726503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.727738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.728225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.728769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.756468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:52.773360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:52.774408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:52.775231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:52.776003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.779423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 315/619] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:19:52.784912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404596a-09fd-4471-aa14-65747dbc76be None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.785533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.826430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.827102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.827390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.827665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.829674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:52.829674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.830966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514aea26-2da3-400b-88e4-9e32676f6d6e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:52.848315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 305/620] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:52.858583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805884-b16f-49e6-b825-43b8ea244ba8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.858887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805884-b16f-49e6-b825-43b8ea244ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.859163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805884-b16f-49e6-b825-43b8ea244ba8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.864897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 306/621] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:52.864897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4805884-b16f-49e6-b825-43b8ea244ba8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:52.940568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:52.947320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea29c5-2a8c-4400-8efe-85d60f17f5e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.981622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4805884-b16f-49e6-b825-43b8ea244ba8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:52.992729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:52.993427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:52.993717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:52.994000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:52.995229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:52.995570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:52.997259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c28908-aa4a-49d4-98b5-2e0a277d09ec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.009268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 316/622] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.010889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.011181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.011465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.021781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.023425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.023938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.024421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.032138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.032403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:53.033635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:53.034421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:53.049082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.050050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:53.050916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:53.055247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 307/623] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:53.056222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.068481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.118692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 317/624] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:53.118692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.187667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1826-e21d-447c-b10a-6f66197c1193 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.214017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.227111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849c8f3-78b5-4eb6-a13e-2631ce6c2bbb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.228327 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.229404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.230249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.230965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.243558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.245516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.263981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.264654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.264939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.265217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.266060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.266374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:53.267447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.268302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.269202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:53.269202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:53.269202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:53.269202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:53.269500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.273092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 308/625] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:53.274257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.289931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 318/626] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.290876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.349886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.395022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496aa1a5-a290-4ca7-83c1-aed7b60420c4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.404772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acab72e-0fcb-41a4-8fda-cf9bb7b9bc3a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.433805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.435330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.435826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.436399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.442175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.442796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.443076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.443348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.444066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.444895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.446568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.450944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:53.459273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.460715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:53.461564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:53.463739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 319/627] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.464648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.465489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 309/628] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:53.466506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.477786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.573662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bea39c-a56d-48ad-b2ad-f99d82aad0ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.581606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d5eaa-673f-47b1-959d-0b4d925ebc25 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.603706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.612365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.613562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.614081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.614489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/9f47a2120e91452ebcaa6e042caed9ed/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.619404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.620315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.620730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.621106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.629846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.631181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=9f47a2120e91452ebcaa6e042caed9ed, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.632317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.633677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:53.647368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:53.648606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:53.649716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:53.656157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 310/629] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:53.656157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.659418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 320/630] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:19:53 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/9f47a2120e91452ebcaa6e042caed9ed/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:19:53.660584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.742896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:53.799938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d8ab-e444-40ff-92bf-9974cd82a941 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.844178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.845122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.845584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.846013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.847003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.847268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331a44-46f1-45e4-84a0-21d84c8d3e2a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:53.847649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.849393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819b034-406d-4a1f-a602-df4a4a249143 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.871067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 321/631] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.872996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb94a-438b-40da-b2f0-88ef40c60abb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.874301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb94a-438b-40da-b2f0-88ef40c60abb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.874738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb94a-438b-40da-b2f0-88ef40c60abb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.876127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 322/632] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:53.877803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cb94a-438b-40da-b2f0-88ef40c60abb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:53.901279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:53.902434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.902847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.903666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.904539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:53.904988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:53.906591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749bfaa-b6a7-4b81-aeff-8d33a70030ad None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:53.920359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 311/633] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:53.925901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:53.927389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:53.927865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:53.953710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:53.955379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:53.995331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.007691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cb94a-438b-40da-b2f0-88ef40c60abb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.026916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.031634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 312/634] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:54.032805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.048087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.048945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.049392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.049766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.123687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.125562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.140077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a82ce-63cd-4dd1-b15d-12cdce2b9cb7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.181165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.182376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.182886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.183365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.196315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.198580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:54.217706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:54.218666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:54.219649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:54.219649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:54.219649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:54.219649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:54.222775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 313/635] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:54.223782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.243643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 323/636] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:54.245693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.263073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.273643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.357061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbb25d-c632-44e5-b5e5-800077263a3b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.369798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daa555-79e8-49c1-b88f-bade73fd2deb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.393563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.394742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.395240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.395742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.406867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.407549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.407836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.408115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.408225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:54.408845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:54.409157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.410567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:54.417683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:54.418648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:54.419461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:54.423613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 314/638] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:54.424691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.425412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 324/638] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:54.429121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.529019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.539136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.547809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce3eb-b503-46a7-98da-ac31e3b454c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.571504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67875fa8-9c6d-44ec-9fc0-de3671ead937 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.586631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.587604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.587805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.588189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.600085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.601926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.615973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.617586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.618067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.618525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:54.621800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:54.622675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:54.623565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:54.623565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:54.623565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:54.623565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:54.623565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 315/639] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:54.624525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.631597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.633475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:54.649157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:54.650230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:54.651174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 325/640] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:54.651791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.752670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ff72b-ef08-442a-8fdc-7309b04c0f0c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.768778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556cf4-3107-4454-84c4-f34a0f4d398a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.795378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.795973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.796258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.796540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.797290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:54.797639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.799005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.801188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.806614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.807460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.807842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.808118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.808846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:54.809164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.809496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 316/641] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:54.810323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.814998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0fdb3-9ba2-4343-8bf7-48df576ac798 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.823634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:54.834640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 326/642] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:54.836427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322bc98-c304-4ca7-9bd2-e9b50a6f768c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.837218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322bc98-c304-4ca7-9bd2-e9b50a6f768c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.837509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322bc98-c304-4ca7-9bd2-e9b50a6f768c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.841458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 327/643] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:54.841458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322bc98-c304-4ca7-9bd2-e9b50a6f768c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.920148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6ae9b-44d8-455b-9c44-48d95c1f2595 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.949960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322bc98-c304-4ca7-9bd2-e9b50a6f768c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:54.957075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.957769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.958056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.958330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:54.959592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:54.959968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:54.961773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:54.979551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 317/644] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:54.980351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:54.991716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:54.992504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:54.992782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:54.993058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.004706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.006362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-70236ac1-a52d-4e86-9c02-da7db01eca5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:55.024800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:55.025759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:55.026832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:55.026832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:55.026832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:55.026832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:55.026832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 328/645] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:55.027199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.027459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.027737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.044635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.044875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:55.046197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:55.091628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea195d-3efc-4e0a-9198-0738b00e324f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.096655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.125457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 329/646] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:55.125457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.137824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.138794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.139221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.139656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.147537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.148982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:55.162765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:55.164259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:55.165130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 318/647] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:55.165717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.239999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369881c-59a0-4ed5-b51a-b35e7ddde168 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.271960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6abf1-2567-4caf-98be-e43ad8d596c9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.277463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.278677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.279147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.279662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.289578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.310939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.311514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.311795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.312069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.312786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:55.313089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.314485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.326332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 319/648] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:55.327126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.337891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.364981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.367079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.440310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e866-bc2d-4c96-a37f-b1220bd06583 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.476127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 330/649] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:55.476943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.477831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.478115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.478390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.481308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.488858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:55.497155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:55.498177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:55.499009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:55.501655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 320/650] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:55.502847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.538351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.572907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.594176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d1b8-3b06-4a5a-bffa-e01ac5ec79cb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.614054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d3a2f-d3d7-47eb-885e-e495ba422f0c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.628679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.629409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.629696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.629993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.630766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:55.631078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.633040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.644532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 331/651] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:55.645388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.660247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.661036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.661318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.661627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.669044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.670509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:55.685610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:55.686559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:55.687497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 321/652] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:55.688094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.762484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6674cd-bef6-4d47-a057-e06d78492029 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.786060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.793427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263ca03-5536-4bee-b585-8e2518283fb7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.796657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.797520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.797807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.798086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.807148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:55.809630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.811524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:55.828769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:55.829921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:55.830781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:55.830781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:55.830781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:55.830781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:55.831005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.831096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 332/653] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:55.831173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.831173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.831173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.831356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.831429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:55.831496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.832834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.842287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 322/654] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:55.843074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.930976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b924f3e-3085-4632-80f7-09c8cdb36c47 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.940452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b645571-39ca-405b-86ef-e0b0a7e6368e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:55.963622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.964270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.964558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.964832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.965697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:55.966011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.967435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc59031-8711-4b6c-a0c8-4901fa268588 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:55.973092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 333/655] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:55.973546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:55.974201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.974304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588698bd-20ec-4990-a129-31d58e3def03 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:55.974480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.974583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588698bd-20ec-4990-a129-31d58e3def03 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:55.974742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.974841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588698bd-20ec-4990-a129-31d58e3def03 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:55.975608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:55.975916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:55.980232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 334/656] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:55.980232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588698bd-20ec-4990-a129-31d58e3def03 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:55.980498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.007691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 323/657] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:56.008651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.028888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.045617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.101364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588698bd-20ec-4990-a129-31d58e3def03 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.121143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7db129-bb1f-4686-8ffa-0d076be2716e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.141892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.142559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.142839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.143112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.143842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:56.144143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.145546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d158-6551-404d-9f1e-6c48287b7060 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.158124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 335/658] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:56.159378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.159662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.159936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.163211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.164011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.164292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.164563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.176789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.178263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.180110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:56.182640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:56.199096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:56.200028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:56.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:56.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:56.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:56.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:56.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 324/659] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:56.201167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.265371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 336/660] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:56.266207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.280896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.316041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.321654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d2c12-c554-4339-beaf-cfa8bdb22d2a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.359694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.360263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.360549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.360822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.372238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ac42f-03d5-48b4-8b61-a0cef73b72da None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.410826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.411745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.411876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.412157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.422671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.424432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:56.439597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:56.440546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:56.446634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:56.447096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.447176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 337/661] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:56.447252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.447322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.521297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.559301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f524c7-573c-42fc-abf0-dd015f77da38 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.567262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 325/662] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:56.567977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.568137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.596943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.597926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.598365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.598780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.599653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:56.600110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.602204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.614441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 338/663] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:56.615431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.680290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069ecc-c19e-4ea3-af29-7e21e95bcebc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.714208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.714998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.715278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.715554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.720798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dedc-b789-4b09-ad4d-e078d7bca00a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.725614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:56.733589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:56.734624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:56.735439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 326/664] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:56.736120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.762048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.762823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.763107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.763378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.764157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:56.764464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.765853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.766859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.784821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 339/665] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:56.784821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.815173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:56.847552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e6d48-5277-40b9-aa78-51ee14a1a85c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.883629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.884324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.884611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.884889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.891193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fffef6-599e-4b5c-a97e-ea55a54a1e6c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:56.896062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.898024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:56.914879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:56.915760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:56.916651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:56.916651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:56.916651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:56.916651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:56.916651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 327/666] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:56.917014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:56.931186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:56.931979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:56.932258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:56.932527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:56.940658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:56.942122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:56.957024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:56.958110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:56.959027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 340/667] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:56.959633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.007281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.027365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3b9d-c126-4602-86b9-59b470576bf2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.051546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.063116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.063773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.064056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.064330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.065063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.065396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.065495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb15fcc-1e01-4c0f-986b-5341bd78a063 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.066757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.075313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 328/668] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.076095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.104482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.105250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.105838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.105967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.106766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=0df73889cb3f4bb58bdb5d674ed5379d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.107078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.108680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.121450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 341/669] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:56 2022] GET /identity/v3/role_assignments?user.id=0df73889cb3f4bb58bdb5d674ed5379d&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.121450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.205462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8357d-6685-41f2-94c9-c4101c040d79 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.249864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.250572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.250858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.251305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.252412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.252725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.254626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.262391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf6965-637d-4349-8167-b865ae2a00df None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.273507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 329/670] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.274455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.277771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.312879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.313579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.313866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.314142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.314987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.315318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.316727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.337269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 342/671] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.338027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.351586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.410821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8027ba-574d-46e6-9098-651a32f98d09 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.458834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.460586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.461652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.462025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.464689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ad3bd-17ba-42fc-a77b-a9755ff6105f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.476809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.478661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:57.500575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:57.501539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:57.502447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:57.502447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:57.502447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:57.502447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:57.503464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.504323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.504661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.505004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.505442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 330/672] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:57.506471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.517791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.519551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:57.535589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:57.536522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:57.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:57.540550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 343/673] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:57.545671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.579390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.639979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.641799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039cb61-c0ae-4867-85ca-01047e753b82 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.685051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.685726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a18f-c464-451a-b317-dae467e5491d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.685840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.686019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.686283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.733971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.735256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.736622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.737115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.738091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.738624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.740327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.756820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 344/674] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.757742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.784333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.786714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.858429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.862481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e935c-0214-4c20-9108-a8d6d753d87d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:57.890092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:57.898734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:57.899095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:57.899388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:57.899666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:57.900459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:57.900792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:57.902238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:57.917560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 331/676] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:57.918746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:57.919615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 345/676] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:57.920831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.025565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493745d4-36f1-4700-b654-7e30e7362785 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.032400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f5d2a-7c46-419c-9002-c958234dc7c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.061116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.062135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.062422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.062702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.063820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=50b29bba6ee54cab9e3f6fdcc841ec77) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:58.064138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.066082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.075115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 346/677] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/role_assignments?user.id=50b29bba6ee54cab9e3f6fdcc841ec77&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:58.076149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.081180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.083144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.083446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.083734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.100939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.102941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-72810ed0-025f-4720-a717-df4c23c04eb4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:58.131286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:58.132344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:19:58.133270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:58.133851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 332/678] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:58.134908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122038a-a272-407c-8287-2eadc5c3afdb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.135193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122038a-a272-407c-8287-2eadc5c3afdb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.135493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122038a-a272-407c-8287-2eadc5c3afdb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.137114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 333/679] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:58.141950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122038a-a272-407c-8287-2eadc5c3afdb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.160143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.165768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.237269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4c9d0-f76c-4f75-9141-eebb9ab7abbc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.275098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122038a-a272-407c-8287-2eadc5c3afdb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.297914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.298103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.298590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.298711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.315511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.317100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.329194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.330231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.330688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.331119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.332108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:58.332565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.334166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0b772e6d-36b1-4c23-8572-7f7a4ba8687d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:58.337147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:58.338170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:58.339050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:58.339050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:58.339050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:58.339050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:58.342887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 347/680] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:58.344249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.344544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.344823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.357375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 334/681] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:58.358234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.370144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:58.371636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:58.443893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.460260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 348/682] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:58.461419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.486352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a56b3-5dc8-49fd-a6cf-57c22f9d19ff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.500951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.536466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.537246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.537644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.538305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.539126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:58.540076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.541827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.564019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 335/683] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:58.564019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.583870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e176fe-443e-4c37-8e7e-cf3d9fddceb5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.622681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.623780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.624254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.624721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.681515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69da45d-4c28-4474-8dd2-a2c225ffe920 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.708804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.711094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.719720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.726777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.727777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.728133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.728506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.743181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:19:58.752383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:58.753364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:19:58.754188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:58.754819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:58.763389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 336/684] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:58.764530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.843280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 349/685] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:19:58.843280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.877605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff171a3-be77-489b-bce5-564b71b84d2f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:58.914459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:58.915677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:58.916159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:58.916629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:58.923837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:58.925535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:58.941725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:58.942688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:58.943555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:58.951093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 337/686] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:58.952227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:58.962814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c1410-7e09-44ec-86db-f75a35310a9a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.005110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.006231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.007027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.007520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.008550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.009101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.010692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.011756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.014133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.030487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 350/687] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:59.031609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.072065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207c241-33cb-4c7b-ba1c-f0e0a790a006 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.107310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.108023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.108313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.108595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.119640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.121100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:59.138969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:59.139848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:59.140740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:59.140740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:59.140740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:59.140740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:59.140740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 338/688] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:59.141447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.145151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2a07-4e9a-439c-9aa8-ef409a4eef40 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.183150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.183820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.184103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.184382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.185274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.185615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.187195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.201492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 351/689] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:59.202486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.245576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1cc75-0233-445b-a817-38b209475f07 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.247989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.265530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.282066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.282961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.283310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.283598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.295052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.296622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.318151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053c1f3-e0fb-45cb-8e73-321778470804 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:19:59.320624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:59.321657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:59.322562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:19:59.322562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:19:59.322562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:19:59.322562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:59.322562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 339/690] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:59.323402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.361607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.362193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.362477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.362750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.363470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.363776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.365151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.381464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 352/691] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:59.381464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.441417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15e02a-8a1b-4c35-9e4d-02629aca6bf0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.476856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.477531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.477824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.478101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.478923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=2b899731a1ef4d9b961492dd78632ac5) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.479235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.480738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d98b99-252a-4e3f-b041-53582a66c2c7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 340/692] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/role_assignments?user.id=2b899731a1ef4d9b961492dd78632ac5&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fa74f-921d-4a71-a992-303012260e69 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fa74f-921d-4a71-a992-303012260e69 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fa74f-921d-4a71-a992-303012260e69 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 341/693] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:19:59.493360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fa74f-921d-4a71-a992-303012260e69 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.507088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c6e2d-09e0-47a2-8b69-948a7db80ac7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.518347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.529263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.553581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.554762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.555272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.555825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.557065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.557604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.559734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e99c5-3280-4bd5-bc80-311152d14ea6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.579457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 353/694] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:19:59.581323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.581843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.582326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.602619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:19:59.604180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:19:59.630698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fa74f-921d-4a71-a992-303012260e69 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.683211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.683862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.684240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.684525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.688577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 354/695] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:19:59.689458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.692816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.696768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:19:59.709782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:59.710693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:19:59.711671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 342/696] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:59.717188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.821553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f9a3c-f0d7-4567-9dd0-2b789458c3b2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.829625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.838811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:19:59.861640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.862600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.862975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.863347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.865737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a86c15-4651-4490-a5dd-d1d2a2c8a683 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:19:59.874305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.877349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:19:59.898484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:19:59.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:19:59.900397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:19:59.903162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 355/697] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:19:59.904193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:19:59.913737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:19:59.914616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:19:59.915050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:19:59.915557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:19:59.916632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:19:59.917169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:19:59.919011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:19:59.934389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 343/698] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:19:59.935615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.035294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78868b30-9d0a-48f8-90b4-8fcc9bd8b71d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.048816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce778b8c-734f-4384-a177-d301e264e052 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.081072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.082230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.082516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.082797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.089118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.090069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.090572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.091045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.092075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:00.092585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.094256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.094920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.096462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.104660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:00.115133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:00.116097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:00.116961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:00.116961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:00.116961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:00.116961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:00.118377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 344/699] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:00.119413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.120627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 356/700] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:00.123057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.143939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.232350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2686e9-fa82-400b-bfdf-e361d6939021 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.240246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf42ad-0d85-446f-901a-0574355d3cc8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.273029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.273714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.273996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.274271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.275101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=9f47a2120e91452ebcaa6e042caed9ed) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:00.275411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.276840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.282977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 345/701] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/role_assignments?user.id=9f47a2120e91452ebcaa6e042caed9ed&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:00.283773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.295573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.296653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.297143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.298268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.299769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:00.300618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.302882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976c341-eec4-4538-95cc-a97508f44b48 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.321212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 357/702] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:00.324287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7617f-d3e9-4a21-95d5-85f8a1bff220 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.324893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7617f-d3e9-4a21-95d5-85f8a1bff220 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.325394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7617f-d3e9-4a21-95d5-85f8a1bff220 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.328554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 358/703] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:00.329624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7617f-d3e9-4a21-95d5-85f8a1bff220 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.364769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.430323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510949d6-c2e3-46e3-a841-010805502bfa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.436031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.471110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7617f-d3e9-4a21-95d5-85f8a1bff220 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.483661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.483823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.484049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.484333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.519669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.520599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.520895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.521613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.522859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.524736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05396808-f596-4b97-a2e4-abb0fb569368 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.548982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 359/704] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:00 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:00.550113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.550405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.551055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.568040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.569634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.570189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:00.571444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:00.649064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.657168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 360/705] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:00.691740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.702050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 346/706] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:00.709105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.794689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:00.851656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009050f-db70-48cd-8304-58af9aa597f2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.861156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae648a9-40bf-471a-8da4-7dbc2a4b1bb2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:00.896853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.899663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.900180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.900862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.902837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.904826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.907826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:00.908620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:00.908902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:00.909176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:00.921697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:00.922702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:00.923580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 361/707] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:00.928542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:00.928980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.930395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:00.951825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:00.952772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:00.953705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:00.953705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:00.953705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:00.953705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:00.964326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 347/708] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:00.965976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:00.983722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.056543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e30b-71f7-4eca-82d9-01f66304bc60 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.097293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe486e-082b-45e8-9464-72c2a1546c64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.098623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.133916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.134601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.134886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.135164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.135883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.136198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.137607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.141842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.142505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.142793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.143076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.143800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.144110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.145490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.151226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 362/709] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.158824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.174151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 348/710] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.180024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.276907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.308597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399ae9-d78b-4976-a72f-16f7cb5df4b8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.359952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cf16c-bbef-4c4a-bbae-565b1959a42e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.361258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.362321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.362826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.363318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.381057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.382905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:01.410683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:01.411627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:01.412540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:01.417239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 349/711] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:01.424319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.441167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.441887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.442173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.442448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.443166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.443477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.444872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.460616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 363/712] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.604519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645cdd0-e7b5-4c7d-91f4-48d6784ab7e7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.645040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.645750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.646011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.646286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.647075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.647447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.648858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9174dee-73c4-4b0d-9baf-206be8b5b6b7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.650677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.662358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 350/713] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.668368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:01.761254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:01.788543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a05e2e-51a9-4e96-9731-92fd7ada1e85 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.789051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a05e2e-51a9-4e96-9731-92fd7ada1e85 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.789125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a05e2e-51a9-4e96-9731-92fd7ada1e85 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.790680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 351/714] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:01.804868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.805185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.805466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.811838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3797-b396-47b7-84d8-f611e0e4dec9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:01.836363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:01.839299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:01.878003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:01.878596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:01.878875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:01.879155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:01.879966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:01.880274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:01.882260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:01.891638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 364/715] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:01.951443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 352/716] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 14:20:01.984236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:02.117846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f443e-24bb-49af-852d-4b7638edb389 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:02.145384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:02.167839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:02.168643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:02.168923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:02.169198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:02.169871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:02.171230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:02.189151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:02.190290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:02.191143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 365/717] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:01 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:02.197362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:02.351270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff2c56-2e01-4b79-b8b1-2b0ca46209e6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:02.385044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:02.396628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:02.397283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:02.397593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:02.397872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:02.398584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:02.398891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:02.400253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3526b-4988-4f78-aed9-860d21825d95 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:02.409989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 353/718] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:02.419079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:02.541485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4203d-2a02-4b16-ab20-f466ea9deed4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:02.584906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:02.585610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:02.585898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:02.586182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:02.586899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:02.587211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:02.588597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9220d-197b-427d-b401-3ecfc0f90009 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:02.593546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 366/719] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:02.783581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79e05c-9202-47bd-b9a6-f4abedd500c1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:02.784233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79e05c-9202-47bd-b9a6-f4abedd500c1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:02.784728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79e05c-9202-47bd-b9a6-f4abedd500c1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:02.786676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 354/720] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:02.792273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:02.792554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:02.792837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:02.817707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:02.818938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:02.876693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:02.905782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 367/721] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:02.930953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79e05c-9202-47bd-b9a6-f4abedd500c1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.066630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79e05c-9202-47bd-b9a6-f4abedd500c1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.110153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.111216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.111645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.112089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.126656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:03.135091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:03.136295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:03.137308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:03.143851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 355/722] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:03.149651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.281125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c04d8b-7111-4537-b660-cf97be92592f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.330271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.330872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.331159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.331435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.332182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:03.332489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:03.333956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0f562-aa4d-491e-9169-f93ea8bd3e54 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:03.342954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 368/723] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:03.348304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.445980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:03.519213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2735b2-1dbe-4002-80ef-01b6c4383cf9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.590982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.591784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.592118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.592396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.607035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994627-7a81-4831-846b-6548277b3857 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.607529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994627-7a81-4831-846b-6548277b3857 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.607632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74994627-7a81-4831-846b-6548277b3857 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.609321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:03.609461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 369/724] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:03.610885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:03.617606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.618156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.618660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:03.637360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:03.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:03.639237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:03.639237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:03.639237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:03.639237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:03.644164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 356/725] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:03.648589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:03.649807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:03.652503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.689043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:03.748967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 370/726] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:03.786992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.836907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:03.843487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a54bf-d9a8-4da1-b87d-f8b24565032c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:03.907279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:03.908621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:03.910086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:03.910706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:03.912841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:03.914171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:03.915728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:03.941317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 357/727] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:03.947287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:03.973160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20f48c-656a-4d46-a935-44537e2b9667 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.035923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.036828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.037015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.037272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.058707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:04.074570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:04.075527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:04.076344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 371/728] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:04.082026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b4843-c2ec-4807-bcb5-d423904651a0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.082186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.120825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.122152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.123146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.123146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.134329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.134998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.137319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:04.154394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:04.155366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:04.156287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:04.158950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 358/729] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:04.166607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.213389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.220124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309bb03-3e00-44b4-839b-3f95c2d58e21 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.262333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.263915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.264633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.265106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.266342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:04.267319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.269130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed517dd-d386-4a65-9f9f-8a8d3c0e51f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.282023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 372/730] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:04.284431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8a76c-108f-4f83-819b-996af50d0609 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.284977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8a76c-108f-4f83-819b-996af50d0609 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.285432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8a76c-108f-4f83-819b-996af50d0609 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.289374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 373/731] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:04.290419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8a76c-108f-4f83-819b-996af50d0609 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.329552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d3ddd-86a0-4ba6-8850-f16d33764f71 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.366893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.368043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.368548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.369019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.370341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:04.370941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.372676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055c9c1-99bf-4744-b6aa-4c7800dde6b8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.395466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 359/732] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:04.396819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecfc-e547-46b2-acf7-ce3d69fd2f97 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.397675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecfc-e547-46b2-acf7-ce3d69fd2f97 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.397960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ecfc-e547-46b2-acf7-ce3d69fd2f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.421072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.429082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-15f0ecfc-e547-46b2-acf7-ce3d69fd2f97 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:04.430576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-15f0ecfc-e547-46b2-acf7-ce3d69fd2f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:04.433892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8a76c-108f-4f83-819b-996af50d0609 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.469216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.470048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.470328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.470601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.483230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.484690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:04.501595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:04.502522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:04.503402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:04.503402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:04.503402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:04.503402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:04.503402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 374/733] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:04.503674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.511527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 360/734] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:04.512689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4d926-e6f2-4107-be06-fc2da7788b5d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.512978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4d926-e6f2-4107-be06-fc2da7788b5d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.513249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4d926-e6f2-4107-be06-fc2da7788b5d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.514530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 361/735] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:04.515204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4d926-e6f2-4107-be06-fc2da7788b5d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.528888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.623196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60441c7e-74b1-44ae-910d-9485559a0ff0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.624910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4d926-e6f2-4107-be06-fc2da7788b5d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.659592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.660459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.660661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.661080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.661191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.661439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.661719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.662015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.662456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:04.663294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.664674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e7b64-7012-4c9c-9024-22496c7a1af6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.677237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 362/736] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:04.678588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.678877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.679157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.681695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.696002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:04.697191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:04.739738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.741641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.762139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.769160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 363/737] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:04.770120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.861504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 375/738] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:04.862385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.886047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b97cf-ffb2-4ab4-baba-d95315fee976 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:04.907302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:04.924680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:04.925356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:04.925652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:04.925935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:04.926894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:04.928245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:04.944965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 364/739] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:04 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:04.946605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:04.974245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af1d4-d54a-41db-bb78-06f6ec743e4a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.024964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.026328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.026614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.026893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.029110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.037536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.039087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:05.056187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:05.057203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:05.058184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:05.061676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 376/740] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:05.063177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.081592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b32a1-582c-417a-92b2-84a627e936c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.120956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.121843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.122087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.122338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.133495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-657c116d-cf36-4901-bc09-de6b80371e89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:05.141414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:05.142547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:05.143531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 365/741] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:05.144351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7752-915a-4d63-9d42-1174902c256e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.144351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7752-915a-4d63-9d42-1174902c256e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.144502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc7752-915a-4d63-9d42-1174902c256e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.146614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 366/742] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:05.146814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc7752-915a-4d63-9d42-1174902c256e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.177393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.181613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33586f5c-6240-4d13-b0fd-108e0fe2459e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.216421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.217537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.217826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.218098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.228747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.230230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:05.247696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:05.248619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:05.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:05.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:05.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:05.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:05.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 377/743] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:05.249830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.265919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc7752-915a-4d63-9d42-1174902c256e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.285723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.306057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.306812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.307096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.307375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.308209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:05.308519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.310186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de1019-f3a2-4db4-a7f6-264c97446289 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.332457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 367/744] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:05.332457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b99f-dffd-4590-a636-45541164d43c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.332457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b99f-dffd-4590-a636-45541164d43c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.332457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827b99f-dffd-4590-a636-45541164d43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.349023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e827b99f-dffd-4590-a636-45541164d43c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:05.350527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e827b99f-dffd-4590-a636-45541164d43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:05.358468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fca-a361-40ca-ba1c-51b53377f68e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.395815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.396779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.397164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.397977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.398743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:05.399030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.400390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.415665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 378/745] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:05.415665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.431556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 368/746] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 14:20:05.433000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.434290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827b99f-dffd-4590-a636-45541164d43c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.535457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd61ce-acde-476b-a407-5921438cb78f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.542544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827b99f-dffd-4590-a636-45541164d43c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.551104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.571995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.572662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.572950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.573229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.573985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:05.574295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.575664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.580281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.580922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.581207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.581533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.591760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 379/747] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:05.591760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.661728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.663441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.669252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.697363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbea4c-236b-4b55-a83d-ef68866cf99a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.732506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.733304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.733619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.733901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.745212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.746696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:05.767612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:05.768513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:05.769418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:05.769418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:05.769418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:05.769418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:05.777717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 369/748] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 14:20:05.777717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.777975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 380/749] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:05.777975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.785068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.881405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94754-5e10-43c2-b355-bae44bb32810 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.883229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67729f-6fa5-4e27-8285-80e315ab3d10 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:05.907483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:05.917559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.918292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.918576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.918860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.924164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:05.925308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:05.925724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.926035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:05.926751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:05.927421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.927836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:05.930225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:05.941460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:05.942363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:05.943270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 381/750] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:05.943753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:05.946785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 370/751] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:05 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:05.948078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.052179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.112531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64c1-ef7c-43da-bbcc-e4be3efe1800 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.116339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fce929-87c7-4937-8fc6-f730d25d1462 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.152593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.153267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.153586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.153871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.154598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.154909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.156008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.156289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.156803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.157088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.158058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.171138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.172448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 382/752] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:06.173192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.173936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:06.200042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:06.200984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 371/753] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:06.201920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.233448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.290941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa483f9-915f-4e69-8274-2628b960062b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.315581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d7fd9-3ee8-4289-96e9-37897646620a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.325919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.326654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.326998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.327330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.328174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.328543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.330053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.330541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.343063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 383/754] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:06.344049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.354660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.355560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.355851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.356133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.364443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.366080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:06.381877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:06.383905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:06.386078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:06.392041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 372/755] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:06.392041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.466309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ac052-c8ef-4272-b6c6-f65ee7db5c21 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.474522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.505850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.506969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.507460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.507959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.508988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.509552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.511304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.512186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e4452-28ce-4fc6-98fe-379ef574940f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.524525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 384/756] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:06.525820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.554065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.554740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.555089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.555377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.556278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=9f47a2120e91452ebcaa6e042caed9ed) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.556594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.558115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.564312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 373/757] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/role_assignments?user.id=9f47a2120e91452ebcaa6e042caed9ed&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:06.565237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.623466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.692695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6e33b-188a-4686-b090-20a8bf18fc03 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.707355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1a85-fc90-4575-9e98-d09b1d6a736c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.755734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.756984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.757510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.757978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.775172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.779686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.793509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.794202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.794487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.794765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.795621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.795936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.797386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:06.800362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:06.801215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:06.802464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:06.806273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 374/758] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:06.812067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.821027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 385/759] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:06.827900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:06.888731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.929656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:06.935173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021ad8-9817-4f23-9747-1b6096387f20 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:06.979514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:06.980911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:06.981306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:06.981654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:06.982466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:06.982779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:06.984228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c4cf4-b124-418f-a26c-a86349fceb34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.107669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2790fa-99bf-404c-b4a7-cc8e55658da0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.148006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 375/760] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.149743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8066-a70a-4c0e-8f91-349c26122b77 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.150029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8066-a70a-4c0e-8f91-349c26122b77 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.150306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8066-a70a-4c0e-8f91-349c26122b77 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.151160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.151962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.152081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 376/761] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:07.152248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.152521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.159877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be8066-a70a-4c0e-8f91-349c26122b77 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.169156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.170648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e994c50a-9ff1-41f0-8661-bc0fff634f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:07.189106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.190093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.190978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:07.190978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:07.190978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:07.190978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:07.190978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 386/762] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:07.192743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.192743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.192743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.212922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:07.214135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:07.287813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be8066-a70a-4c0e-8f91-349c26122b77 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.290820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 387/763] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:07.292124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.300554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.327839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.328866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.329445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.329996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.331090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=b1f2a519e5cc42a993a6007174343edf) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.331614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.333233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.340488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 377/764] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/role_assignments?user.id=b1f2a519e5cc42a993a6007174343edf&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.345528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.389799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.408406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e502f9-08fd-40d9-a9ef-cd15c7908063 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.448642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.449511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.449872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.450315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.451030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.451411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.452886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.467040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 388/765] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.472710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.554165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.588568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e7c17-d6da-4df2-8679-3e66a1d5b482 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.588760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460edae2-2bf1-41fd-b252-8f2a8ab42275 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.626749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.628095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.628383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.628657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.629304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.630679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.646331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.647918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:07.648761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 378/766] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:07.655698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.677000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.678162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.678620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.679041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.686526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.688289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:07.703927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:07.705442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:07.706552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:07.708695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 389/767] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:07.714060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.831626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0edf-b349-47fa-bcc9-1abe32b8059f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.851639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.863888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:07.875825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.876887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.877585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.878029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.879384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.879849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.882122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.884355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d22d5-fdfb-4e95-9bd5-9f786cb7c69a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:07.909239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 390/768] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.915659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:07.948737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:07.949469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:07.949742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:07.950019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:07.950728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:07.951042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:07.952391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:07.968456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 379/769] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:07.971609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.080137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf37f-38d6-4207-8bd5-1b96f96777ac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.104755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4155be2-2060-4506-8e79-b8614da226a1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.146807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.148565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.149053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.149688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.150892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:08.151489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.153110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.160503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 380/770] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:08.167041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.170033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.170940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.171222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.172288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.174697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=a1fb1196840240b4aa05a2f83d4c7ee7, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.176125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.188585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.198276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 391/771] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:07 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:08.284586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea9e63-c815-442b-b651-bfc6c74a838d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.322038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.323143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.323630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.324107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.325105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] RBAC: Adding query filter params (interface=public, service_id=e1939c7b415149b385fccba3a01808ac, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:08.325666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.327319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91548a-134e-417f-a2ce-a770553ade94 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.336751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 381/772] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/endpoints?service_id=e1939c7b415149b385fccba3a01808ac&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:08.344691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.421011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.440087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85c24c-9e89-4e66-bc96-3420e8338892 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.440844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85c24c-9e89-4e66-bc96-3420e8338892 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.441322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85c24c-9e89-4e66-bc96-3420e8338892 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.443015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 382/773] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:08.452654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.453256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.453795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.486529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:08.486836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256e3cc-b6ec-4be6-b76a-7d5933642074 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.487745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:08.530104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.530770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.531046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.531317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.532002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.533521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.540409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 392/774] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:08.594941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 383/775] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:08.633700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.786823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57d09-0b29-4855-bb08-fcc8229e0359 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:08.801950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:08.862368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:08.863241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:08.863529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:08.863857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:08.864521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:08.866113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:08.883889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:08.884885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:08.885759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 393/776] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/services/network => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:08.890984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:08.904776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.032040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292f2ed-9695-4e76-9802-17db63ed57f5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.080515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.081108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.081448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.081726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.082443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.082747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.084108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68683867-08d9-4c75-a0a9-e8a82c1b2b02 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.093449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 384/777] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.098968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.105386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff26b-a7e9-4f1b-ba42-e584a8b3a2a2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.105844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff26b-a7e9-4f1b-ba42-e584a8b3a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.106017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff26b-a7e9-4f1b-ba42-e584a8b3a2a2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.107517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 385/778] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:09.116550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.116882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.117166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.149765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:09.150956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:09.286699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 386/779] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:09.292800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b1dca-8495-462a-8e03-7d09a247d064 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.327414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.328083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.328366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.329652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.331982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.335466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.342422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.342740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.344145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.357533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 394/780] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.365054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.469548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.478083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1098-320f-4eeb-8381-aaf054d92da0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.514067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.515264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.515700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.516124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.528821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:09.537267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:09.538287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:09.539122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:09.541965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 387/781] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:09.548035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.597897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6e3c-daa1-47d1-8361-df6eac2ab1ae None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.645714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.699627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50464f-bb2c-4365-abb4-b1ec469864e7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.704089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.704767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.705050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.705326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.706153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] RBAC: Adding query filter params (interface=public, service_id=ea0fee6f1c5d46e7a223cc73087ba471, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.706465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.707835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.726532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 395/782] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/endpoints?service_id=ea0fee6f1c5d46e7a223cc73087ba471&interface=public®ion_id=RegionOne => generated 194 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.732013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.773916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.774587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.774871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.775150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.775897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:09.777667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.779315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b8dcd-88dc-4fc5-a72b-28782768bc05 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.797480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 388/783] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:09.798835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b879c-4834-4eaf-93d1-efff9ca00999 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.799121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b879c-4834-4eaf-93d1-efff9ca00999 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.799394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b879c-4834-4eaf-93d1-efff9ca00999 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.801783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 389/784] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:09.802523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b879c-4834-4eaf-93d1-efff9ca00999 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:09.875664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8046-4e59-4d45-972d-6071d4cd4bc4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.877908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.912955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.913839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.914220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.914591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.915377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:09.916937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ee40c-b99b-4c75-bffd-cc4cec6ae93f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:09.922866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 396/785] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:09.923134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b879c-4834-4eaf-93d1-efff9ca00999 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:09.924330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9402f-2566-451e-9895-dfd411869453 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.924716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9402f-2566-451e-9895-dfd411869453 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.925094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9402f-2566-451e-9895-dfd411869453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.942696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-60d9402f-2566-451e-9895-dfd411869453 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:09.944719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-60d9402f-2566-451e-9895-dfd411869453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:09.964574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:09.974964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:09.975816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:09.976102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:09.976382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:09.996338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.000887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:10.030582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:10.031539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:10.032422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:10.032422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:10.032422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:10.032422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:10.039963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 390/786] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:10.047508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.074242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 397/787] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:10.113011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9402f-2566-451e-9895-dfd411869453 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.182521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513dcd6-2164-4834-9538-12626fbc73ab None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.224781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.225923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.226224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.227054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.227626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.228671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:10.229272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.231025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.232285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9402f-2566-451e-9895-dfd411869453 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.253411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 391/788] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:10.273654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.276024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.276830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.277108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.277407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.290062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:10.298614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:10.299554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:10.300364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 398/789] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:10.306260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.357094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.478557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aae310-afe2-4f66-9b7c-97ef78d899b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.522912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d553a-6e4f-4874-96cf-1fba38b764b8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.542330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.543267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.545302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.545758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.546945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:10.547840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.550123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.563731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 399/790] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:10.569394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.592596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.593481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.593762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.594033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.601968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.603039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.603402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:10.621835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:10.622854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:10.623817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:10.627642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 392/791] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:10.637356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.723759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54853a-491d-49b1-aab6-d04f65969c63 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.761441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.762193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.763921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.764495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.765024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.768575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b7365-061d-414e-ad29-60a07e36dc69 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.778045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.779894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8950cf8a-dbce-421c-9aa0-c259ca3425ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:10.799535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:10.800474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:10.801395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:10.801395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:10.801395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:10.801395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:10.803313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 400/792] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:10.804749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bca2-ed9a-4831-9a63-322ce42ae2e6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.805190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bca2-ed9a-4831-9a63-322ce42ae2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.805669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4bca2-ed9a-4831-9a63-322ce42ae2e6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.807337 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 401/793] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:10.808555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4bca2-ed9a-4831-9a63-322ce42ae2e6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.813630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.814317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.814603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.814882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.815733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:10.816043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.817516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547c5da-9bb3-48ff-ae87-1f9f40f34280 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.838532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 393/794] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:10.839884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.840171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.840449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.858227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:10.859528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:10.886949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:10.922170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4bca2-ed9a-4831-9a63-322ce42ae2e6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:10.934944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 394/795] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:10.938473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:10.962665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:10.963444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.963726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.964110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.964880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:10.965190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:10.966600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932b353-9474-4f9a-a8fe-69ce3cb51053 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:10.979041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 402/796] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:10.980285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816844fe-0ed6-40c7-9c13-6a86cf4de465 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:10.980572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816844fe-0ed6-40c7-9c13-6a86cf4de465 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:10.980850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816844fe-0ed6-40c7-9c13-6a86cf4de465 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:10.982325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 403/797] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:10.983031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816844fe-0ed6-40c7-9c13-6a86cf4de465 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.038263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.055860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c178d13-e730-454a-a10a-b68ff1fb3673 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.090740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.091529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.091811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.092083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.092351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816844fe-0ed6-40c7-9c13-6a86cf4de465 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.106257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.107695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.125418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:11.127064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.128110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.129178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:11.129178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:11.129178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:11.129178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:11.133193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 395/798] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.134043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.136700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.137700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.137983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.138258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.149754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-32ef0099-ce7b-4e54-8818-a6fcde8cbf13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:11.157347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.158307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:11.159128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 404/799] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.159803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.160460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.160460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.180110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:11.181734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:11.249384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c0f4-e5e4-4ea3-9186-189dfa2801cb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.261398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 405/800] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:11.261398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.285126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.286193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.286305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.286553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.291393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.298242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.299999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:11.316134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.317194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 396/801] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.318197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.375222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0aad9-916d-46f9-b9fb-34541877a757 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.396201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.414410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.415089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.415377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.415659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.416393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:11.416705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.418118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.431702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe11d-9538-4bd5-9df3-b444a08ddf45 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.433668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 406/802] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.434302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.469620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.470282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.470564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.470839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.471582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:11.471891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.473245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.486183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 397/803] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.486183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.542781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd3c4f-2edf-48b3-9481-47a36a25cbcc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.547983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.578478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.579286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.579571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.579849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.580503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.581902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.594637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.595569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:11.596385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 407/804] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.597206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.599766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51863675-f6a4-4206-8660-57dd69345028 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.635727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.636540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.636828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.637104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.638058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:11.638374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.639797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.646137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.663213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 398/805] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.663213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.707279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9363d-23e0-4a4d-8982-5915160f11cf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.739995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.741324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.741636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.741913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.748702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.750172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:11.764420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.765304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:11.766933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:11.767367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27119a-50a2-49e1-acf7-1c28d3a5be20 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.767442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 408/806] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.767442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.786255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.805995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21841357-9213-4195-81dd-c68026b7016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.806719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21841357-9213-4195-81dd-c68026b7016d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.806999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21841357-9213-4195-81dd-c68026b7016d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.807276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21841357-9213-4195-81dd-c68026b7016d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.817541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21841357-9213-4195-81dd-c68026b7016d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.819063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21841357-9213-4195-81dd-c68026b7016d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-21841357-9213-4195-81dd-c68026b7016d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:11.836924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:11.837912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 399/807] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:11.839452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21841357-9213-4195-81dd-c68026b7016d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.879675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47a8a1-ed27-4ff4-bfa3-f081c9142b2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.882717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:11.917611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.918287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.918570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.918849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:11.919555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:11.919863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:11.921237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:11.926302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 409/808] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:11.927254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:11.951143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21841357-9213-4195-81dd-c68026b7016d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:11.991543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:11.992339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:11.992624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:11.992922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.003998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.005589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:12.024927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:12.026032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:12.031975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:12.031975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:12.031975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:12.031975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.031975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 400/809] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:12.032334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f77ed-929d-4db1-a6c2-e0e65f946fab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.032421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.052547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.068344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.069117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.069445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.069777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.070569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.070879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.072361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.084954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 410/810] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.086432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.118128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.138354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c00da-67a5-43ad-88b8-b129a50e917a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.174798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.175479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.175845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.176123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.177220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.177560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.178989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.196262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 401/811] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.196262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.199778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790bcea-a937-4c1e-9923-f3289e50e705 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.233543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.234206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.234488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.234763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.235560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.235867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.237304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.242437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 411/812] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.242997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.278892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.299866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd275a-bba3-498f-bf46-fa531beb8fe1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.332743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.333348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.333639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.333913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.334131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.335228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.335474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.336859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.343933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7923b-29e8-49b8-958a-b6e968528fa8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.348420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 402/813] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.349189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.376828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.378008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.378488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.378972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/9f47a2120e91452ebcaa6e042caed9ed/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.382916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=9f47a2120e91452ebcaa6e042caed9ed, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.385654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8270-8d57-4607-a907-c4283a3e4a2b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.401462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 412/814] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:12 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/9f47a2120e91452ebcaa6e042caed9ed/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:12.402755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c83bc-5f58-4a52-8da4-219031b8dfe7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.403043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c83bc-5f58-4a52-8da4-219031b8dfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.403321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c83bc-5f58-4a52-8da4-219031b8dfe7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.404795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 413/815] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:12.405360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c83bc-5f58-4a52-8da4-219031b8dfe7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.470316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725423a-effe-4539-a82d-0880aa5d469d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.512315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.513684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.514580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.515021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.524973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c83bc-5f58-4a52-8da4-219031b8dfe7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.525835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.527419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.531085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:12.545049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:12.546295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:12.547422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.550588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 403/816] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:12.551200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.571173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.571829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.572159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.572437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.573264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.574644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841ffcd-394f-4db8-a20f-4ac5f6228bc6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.597414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 414/817] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:12.598494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.598795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.599060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.620445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:12.621790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:12.624508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.705609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad665af-9a3a-46d5-adf7-9e7b46ea3afb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.713383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 415/818] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:12.713383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.763596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.764477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.764765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.765040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.776166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.777675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:12.792585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:12.793552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.794414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 404/819] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:12.794886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.848488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689d57-49f7-44ff-aff8-6e9d0fb72ec1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.869299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.895863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.897285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.897976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.898881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.899931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:12.900542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.903171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.905901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:12.926801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1497f7-08d6-44a3-9de0-df92c3287c73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:12.930572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 416/820] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:12.932000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:12.969252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250755dc-c231-449c-8375-b62708804609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:12.970808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250755dc-c231-449c-8375-b62708804609 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:12.971201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250755dc-c231-449c-8375-b62708804609 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:12.971583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250755dc-c231-449c-8375-b62708804609 None admin] PATH_INFO: `/v3/services/network` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:12.972374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250755dc-c231-449c-8375-b62708804609 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:12.974035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250755dc-c231-449c-8375-b62708804609 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-250755dc-c231-449c-8375-b62708804609 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:12.987734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:12.988993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 14:20:12.990040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:12.992362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 405/821] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:12.998265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250755dc-c231-449c-8375-b62708804609 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.123107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250755dc-c231-449c-8375-b62708804609 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.150628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.163935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.164896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.165323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.165808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.184419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75c3a8-df13-40ff-9c2b-29eb31aa2ff7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.222155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.222933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.223286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.223636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.224725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.224999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.226591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.239915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 417/822] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.241309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.243009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.244874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.296725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.348397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564bb1e-b41a-4268-8afa-81dad32edd55 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.357198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 406/823] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:13.358733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.385204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.386240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.386934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.387424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.387865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.388803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.389272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.390985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.407668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 418/824] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.408881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.480052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8d6ab-8b92-4176-a8eb-a8ed7003d4ff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.517772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.518656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.518936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.519211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/b1f2a519e5cc42a993a6007174343edf/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.536006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=b1f2a519e5cc42a993a6007174343edf, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.537535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.544241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca90e-213a-4097-b241-fd8ac6c1c283 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.561571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 407/825] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:13 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/b1f2a519e5cc42a993a6007174343edf/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:13.562897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.565731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.590870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.592406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.592700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.592979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.607181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.608797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:13.642749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:13.643691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:13.644570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:13.644570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:13.644570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:13.644570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:13.653889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 419/826] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:13.659152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.691849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:13.726901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2ca81-d5d5-4460-87fe-a7eae29ef549 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.771603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.772993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.773671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.774147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.775052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] RBAC: Adding query filter params (type=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.775572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.777807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.785040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 408/827] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:13.790519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.833746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3a620-2d24-4036-a914-6646c2e4cd24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.881165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:13.882509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:13.882801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:13.883075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:13.883815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:13.884122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:13.885597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:13.898582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 420/828] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:13.904232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:13.933397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550714cd-a03f-4b09-acd2-cb684f8a4239 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:13.956377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.003208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.004173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.004632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.005084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.006051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.011039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.029651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.058896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 409/829] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:14.066298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b49e6-21a3-4760-b79b-ee20cfdd4078 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.122409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.123705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.124075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.124563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.135242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.137427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:14.162381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:14.163339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:14.164310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 421/830] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:14.169518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.309986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5feae-f825-41b2-8ca2-92d7f750bb5b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.348529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.349187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.349517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.349788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.351100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:14.351409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.352826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad66115-dd0b-4a96-a21d-e5547dd66dee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.366933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 410/831] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.372357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.497158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.518012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c76a3-963e-49b1-8343-2e11559c3b9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.557159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.557854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.558138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.558418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.559255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:14.559565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:14.560993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a148e2-0ae6-408b-8b0e-8e4f86066c38 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:14.567363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 422/832] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:14.628086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365abe-a0ea-422a-9d0a-d7918812af9a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.628384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365abe-a0ea-422a-9d0a-d7918812af9a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.628660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365abe-a0ea-422a-9d0a-d7918812af9a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.630683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 411/833] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:14.636541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd4a0-b418-41b0-8bf0-2535d93e79ce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.636929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd4a0-b418-41b0-8bf0-2535d93e79ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.637283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd4a0-b418-41b0-8bf0-2535d93e79ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.656094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b82bd4a0-b418-41b0-8bf0-2535d93e79ce None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:14.657608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b82bd4a0-b418-41b0-8bf0-2535d93e79ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:14.735457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 423/834] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:14.760471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365abe-a0ea-422a-9d0a-d7918812af9a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.822852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:14.869719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746c9a7-f5ee-494d-b9ae-0e7f2bd7ae31 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.870163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746c9a7-f5ee-494d-b9ae-0e7f2bd7ae31 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.870755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746c9a7-f5ee-494d-b9ae-0e7f2bd7ae31 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.873157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 424/835] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:14.875205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365abe-a0ea-422a-9d0a-d7918812af9a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:14.879067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.879354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.879633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.896725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:14.898515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:14.913979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:14.914923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:14.915308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:14.915705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:14.927542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:14.936213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:14.937369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:14.938420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:14.940552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 412/836] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:14.947248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:14.967789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 425/837] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:14.989773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.000640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.063670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c80e4-0f6b-435c-b14d-220938af76aa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.079269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.095725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaff6bb-40ca-4a11-824e-3384d9494fcf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.104128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.104961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.105496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.105881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.106733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:15.107140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.108621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.118093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 413/838] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:15.123510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.132108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.133177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.133494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.133774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.134431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.136006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:15.148126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:15.150025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:15.151895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:15.154112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 426/839] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:15.157377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.234274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.235796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae461b24-49e6-436f-891d-d1e0852c79b2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.260331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199711b-cd77-460d-9624-be1314e8747d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.271435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.272487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.272746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.273118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.284571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.286150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.297035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.298020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.298510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.298962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.299841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:15.300325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.302629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:15.305733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:15.306574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:15.307446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:15.307446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:15.307446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:15.307446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:15.314358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 414/841] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:15.314358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.314700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 427/841] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:15.315881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.344098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.421278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059e196-bf1b-443e-98ac-4b3299d45458 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.434873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204375-e7d1-4e65-a56d-a16b42e20c29 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.456978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.457935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.458398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.458857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.459796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:15.460304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.461874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.469825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.469825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 428/842] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:15.475581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.476278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.476567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.476847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.477648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:15.477961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.479383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.494460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 415/843] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:15.500226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.619266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a763d4c-b67c-4904-aaf7-445b4d87a38c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.660673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.661508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.661791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.662067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.670990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.672442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:15.687819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:15.688757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:15.689751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 429/844] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:15.696602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.794457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.812249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a13bb4-d1b8-4de3-ba63-7503ab1fe6e6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.849417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:15.851058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.851547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.852020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.853057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:15.853609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:15.855227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9568e-03e2-448d-af7d-db1306cc8f36 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:15.873931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 416/845] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:15.882689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:15.942192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:15.986018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ad4ad-7907-46b7-aee7-dfedb9922ae1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:15.987576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a67a-5eb5-45ac-94f1-f5982d2a9386 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.988072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a67a-5eb5-45ac-94f1-f5982d2a9386 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.988615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3a67a-5eb5-45ac-94f1-f5982d2a9386 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:15.990505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 417/846] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:15.997893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:15.998171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:15.998453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.016468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:16.017780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:16.020957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.021655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.022000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.022283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.097357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 418/847] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:16.106825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.108300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.116693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.173733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.215642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.223213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 430/848] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:16.230701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.246768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023b976-1426-4658-932d-fbebe1c1249b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.292416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.293615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.294104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.294699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.307283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:16.315891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:16.316805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:20:16.317680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:16.325156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 419/849] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:16.326420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.355954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c1fec-aeaa-4fdc-9658-75deab42bc51 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.394160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.395332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.395847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.396349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.408625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.410438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:16.434099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:16.435846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:16.438315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:16.438315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:16.438315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:16.438315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:16.439860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 431/850] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:16.440696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.457149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.472421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc8ab8-5e6e-4456-84d1-6f6c21412d5f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.511241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.512324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.512812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.513369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.514423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:16.514952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.516559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.519386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.532049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 420/851] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:16.533261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.551952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244d988-2e18-44a4-b481-ff9ecb860492 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.590199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.591112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.591625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.592040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.592909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:16.593496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.595122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.612616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 432/852] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:16.613761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.644573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f2221-b27f-44a4-8ac8-5100c07d6a27 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.684977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.685929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.686413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.686789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.697029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.700086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.701879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-13115858-be49-4df9-aceb-e529a65cf6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:16.721756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:16.722673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:16.723535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:16.723535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:16.723535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:16.723535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:16.726184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 421/853] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:16.727742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d603-ef16-4e20-ad1b-ca9128cf4d36 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.728258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d603-ef16-4e20-ad1b-ca9128cf4d36 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.728738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d603-ef16-4e20-ad1b-ca9128cf4d36 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.729560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c711c3-1769-4242-adb6-853a3414138e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.730745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 422/854] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:16.734681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160d603-ef16-4e20-ad1b-ca9128cf4d36 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.765810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.766599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.766891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.767170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.774301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.775822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.788284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-beeeefc9-c4a9-496c-aeef-a1748906bb63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:16.790896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:16.791816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:16.792694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 433/855] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:16.794139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.794139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.794139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.811431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:16.812621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:16.849360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160d603-ef16-4e20-ad1b-ca9128cf4d36 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:16.885557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 434/856] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:16.885557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.888337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:16.889213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:16.889684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:16.890157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:16.891038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:16.891509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:16.893027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:16.908998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 423/857] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:16.908998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:16.956957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:16.996960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f46e-d6f4-4a19-983f-20b51bef2953 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.021800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119962ff-2657-48cd-99d4-7de3ca358f47 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.032726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.032989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.033569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.034045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.034283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.035110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.035491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.037059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.049498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 435/858] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.055605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.060418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.061346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.061746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.062128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.062902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.064365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:17.077803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:17.078970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:17.080027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:17.086261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 424/859] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 14:20:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:17.086261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.176018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea3d57-374a-48c2-997f-3f630656c25a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.194411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b7b2e-3d33-452a-a9cb-9dcc1a4d30d0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.200643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.212687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57083ae6-f828-45af-a901-540556c19afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.213465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083ae6-f828-45af-a901-540556c19afa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.213751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083ae6-f828-45af-a901-540556c19afa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.214030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57083ae6-f828-45af-a901-540556c19afa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.221704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57083ae6-f828-45af-a901-540556c19afa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.223135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57083ae6-f828-45af-a901-540556c19afa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.236216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.236876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.237163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.237470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-57083ae6-f828-45af-a901-540556c19afa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:17.237859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:17.238760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:17.245385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:17.245996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=9f47a2120e91452ebcaa6e042caed9ed) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.245996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.246130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 436/860] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:17.246130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57083ae6-f828-45af-a901-540556c19afa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.246251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.255219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 425/861] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/role_assignments?user.id=9f47a2120e91452ebcaa6e042caed9ed&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.255219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.312614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.384491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57083ae6-f828-45af-a901-540556c19afa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.393732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5240893-c74c-4cb4-a1de-b98f88b2a311 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.434924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.435849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.436134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.436415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.437139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.437575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.438950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.444882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 437/862] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.445025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.445591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.445882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.446164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.447022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.447338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.448778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.453397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.469142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 426/863] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.474391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.517782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.587482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.587930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb0dfa-1ff3-43cd-a8cd-ca4bc871cfb0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.595648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94bc4c-f3b7-4293-9d57-0c9b444d1bbf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.627422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.628096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.628383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.628663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.629402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:17.629718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.631736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4106dc-1296-4b88-a7bc-b1cfe58d13ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.636244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 438/864] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:17.637612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c268-087d-447b-97ef-3996bffcf460 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.637904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c268-087d-447b-97ef-3996bffcf460 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.638182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973c268-087d-447b-97ef-3996bffcf460 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.639588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 439/865] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:17.646173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.647846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.648191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.648307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.648679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.648873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.649304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.669688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:17.670904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:17.744722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.747053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.757363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 440/866] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:17.776901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.831276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.867338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:17.876589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 427/867] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:20:17.881426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:17.909715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823db0b-47f2-40d0-98e6-dc5bbe154c1a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:17.958030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:17.958961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:17.959272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:17.960048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:17.960949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:17.962939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:17.982268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 441/868] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:17 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:18.014062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67fb61-b607-497e-adb3-874e38dc1106 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.062760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.063621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.063909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.064184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.076807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.078329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:18.098298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:18.099248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:18.101534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:18.101534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:18.101534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:18.101534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:18.101534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 428/869] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:18.106633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.214093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.217933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f627c6-04f7-41d6-bddb-9c4225f63266 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.254352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.255019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.255300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.255574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.256818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:18.257126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.258618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.270416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 442/870] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:18.276461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.336029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.394713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce125a0-6d2a-4c7a-a155-437f8743c0f5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.432008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.432802 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.433085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.433398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.440680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.442231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:18.463398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:18.464354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:18.465221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:18.466224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 429/871] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:18.471995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.579409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f6a2d-083b-422b-80e2-5a8fbe92afd5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.614010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.614678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.614966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.615240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.616552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:18.616861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.618316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74a96d-f45b-4b5d-bb84-e260ccb98398 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.631798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 443/872] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:18.637600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:18.745796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112b933-a06a-46a6-a85f-88be9d4aaaa0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:18.748546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:18.783148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:18.783807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:18.784095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:18.784375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:18.785205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=b1f2a519e5cc42a993a6007174343edf) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:18.785548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:18.787034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e4c64-e974-46d4-a8e5-dc94ab8b0b67 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:18.793891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 430/873] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:18 2022] GET /identity/v3/role_assignments?user.id=b1f2a519e5cc42a993a6007174343edf&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:19.401892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fec88-d8ad-43da-8737-83fe1331225d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.402898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fec88-d8ad-43da-8737-83fe1331225d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.403399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fec88-d8ad-43da-8737-83fe1331225d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.410488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 444/874] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:19.421983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.422508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.422980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.441510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:19.442967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:19.510927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 431/875] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:19.533979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fec88-d8ad-43da-8737-83fe1331225d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.558097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.644938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fec88-d8ad-43da-8737-83fe1331225d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.681148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.681908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.682188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.682463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.683119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.684484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-97d72fc2-7a12-4d04-a02c-dac4fa3cf4cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:19.697463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:19.698475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:19.699294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 445/876] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:19.706761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.732214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9099-02a9-498e-a8e3-97b42055a3b6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.732534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9099-02a9-498e-a8e3-97b42055a3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.732757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9099-02a9-498e-a8e3-97b42055a3b6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.734458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 446/877] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:19.742899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.743292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.743664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.762726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:19.764137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:19.825367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:19.829129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044fde-cd31-45e2-aa43-c1b9707f3001 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:19.852682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 447/878] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:19.871088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:19.872058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:19.872547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:19.873045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:19.874006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:19.874537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:19.875890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.878878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:19.885042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 432/879] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:19.891039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:19.893140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.002082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00db0a-bd4e-4dcf-b813-6a8f5611f54f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.012283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a747ac-19f7-4417-92f1-888f2ffe9a11 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.042625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.043747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.044229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.044727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.045672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.047230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef530885-a58a-4798-b6a7-758b4292f8e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.059012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.059686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.059973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.060250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.061058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:20.061494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.062767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590934c6-826c-400f-815d-e5c52be7ecd3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.072295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 448/880] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:19 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:20.073932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 433/881] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.216474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34930-dd28-4d02-92f5-ae0f0c269e2c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.216838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34930-dd28-4d02-92f5-ae0f0c269e2c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.217051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b34930-dd28-4d02-92f5-ae0f0c269e2c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.218819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 449/882] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:20.224636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.225033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.225290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.243252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:20.244445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:20.268817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.328219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 434/883] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:20.352243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b34930-dd28-4d02-92f5-ae0f0c269e2c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.493065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:20.524015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b34930-dd28-4d02-92f5-ae0f0c269e2c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.559244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.560050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.560331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.560609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.561269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.562733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:20.575748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:20.576760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:20.577646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 450/884] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:20.583406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.690611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1a934-d6a7-419d-bc67-5d72c14e573f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.726818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.728268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.728768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.729240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.730337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:20.730857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.732413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.737959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 435/885] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.749478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.852209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bed1-3705-4d54-9d4b-872aeb4728c4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:20.887723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:20.888678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:20.889122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:20.889573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:20.890426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:20.890901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:20.892413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc457c-64f1-43bc-b72a-e3677b5d6ce4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:20.898573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 451/886] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:20.904553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:20.923334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.012351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a1d6-02d6-4af2-b1d9-75e8f5d3c142 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.022253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cabaf-f3ee-489a-a4b4-b22e503b0e95 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.022565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cabaf-f3ee-489a-a4b4-b22e503b0e95 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.022808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cabaf-f3ee-489a-a4b4-b22e503b0e95 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.027865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 452/887] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:21.029275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.029586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.029863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.044793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.047371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:21.048807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:21.049282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.050055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.050456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.051007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.052593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] RBAC: Adding query filter params (interface=public, service_id=4c379280dae241c59b868a97f7cde3f7, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.053441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.054829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.062042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 436/888] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:20 2022] GET /identity/v3/endpoints?service_id=4c379280dae241c59b868a97f7cde3f7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.067183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.133450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 453/889] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:21.154955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.169094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.191217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97880d-0ce7-44ba-ba23-67dfbcbca35a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.228905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.229760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.230135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.230479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.231324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.232854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.245363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 437/890] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:20:21.283404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f0723-5b6c-4cb4-9c67-252af2a79a64 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.318185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.325389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.326782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.327063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.327334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.343965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:21.353196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:21.354227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:21.355065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 454/891] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:21.360985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.478123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce47d3-6d03-41d7-aba7-2fa42e1a69d3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.513228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.514264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.514752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.515253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.516197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.516706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.518296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.529746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 438/892] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.535640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.638244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301dfe-0c68-4563-bf87-280011d14e23 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.673241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.673995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.674279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.674556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.686033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.687473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:21.705128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:21.706096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:21.707199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:21.707199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:21.707199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:21.707199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:21.707199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 455/893] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:21.714900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.724855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.824465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ee3b-bca6-4f1a-887f-c5ae381c9c1b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:21.859818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:21.860483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:21.860769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:21.861046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:21.861801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:21.862114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:21.863487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:21.876261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 439/894] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:21.882636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:21.932781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:21.988201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dc81f-16d0-409d-8ba8-cb7e6fa0e21d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.022285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.023403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.023861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.024308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.032488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.034159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4d6c6fc7-41e5-4984-8946-9ca21e3eca1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:22.049866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:22.050842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:22.052007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:22.058741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 456/895] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:22.060436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.130094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.170959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a0ff-29c0-422a-b2e7-089094b2a36d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.192217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efc72-b6da-418b-8d00-eb3cc60d0d3e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.192473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efc72-b6da-418b-8d00-eb3cc60d0d3e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.192700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efc72-b6da-418b-8d00-eb3cc60d0d3e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.195029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 457/896] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:22.199964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.200245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.200522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.207661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.208333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.208619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.208889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.209821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:22.210134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.211634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.217849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:22.219039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:22.232038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 440/897] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:22.237803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.301596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 458/898] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:22.320764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.355792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.358304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3af41-cab0-4908-9902-6e65e758ffc8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.387653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.399626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.400270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.400565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.400840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.436184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089a731-5f31-467c-9ac1-be96ba0ca700 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.472844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.473469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.473754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.474031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.474794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.475227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.476195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d92fa-71cd-4354-a429-75d611b1a01a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.476665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.492456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 459/899] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:22 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:22.493938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66147da-4db1-4bd2-a5eb-f32213920714 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.494287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66147da-4db1-4bd2-a5eb-f32213920714 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.494627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66147da-4db1-4bd2-a5eb-f32213920714 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.500448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 460/900] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:22.502069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.502416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.502773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.523950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:22.526446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:22.619163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 461/901] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:22.629001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 441/902] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:20:22.637394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.645497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.659692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.678224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.753883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26fb6-4a36-46c1-8d38-9252ecb10287 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.778378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ef022-428c-408f-bb37-6088dcbc1036 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.796880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.797980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.798265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.798694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.812428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.814229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.821389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.822206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.822492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.822872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.823529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.824981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:22.835194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:22.836064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:22.836941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:22.836941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:22.836941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:22.836941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:22.837610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:22.839627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:22.841560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 442/904] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:22.842593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 462/904] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:22.842684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.849355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:22.918064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.945883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:22.951139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335be972-6ccf-4880-b511-d4da96a4fc96 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.962454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6d687-2112-4be3-8e87-4055f14c9426 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:22.986596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:22.987378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:22.987724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:22.988072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:22.988970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:22.989208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:22.991514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:22.997215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 463/905] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.002433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.005850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.006523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.006801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.007073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.007817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.008129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.009530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.028020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 443/906] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.029532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.140858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a977e-77cd-4ab2-8087-154a5b4f9a1d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.142130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b5b24-e53b-44a8-8a9e-3e8cfa87580d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.178884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.179844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.180092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.180434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.188057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.189356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.189788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.190592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.191054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.191777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.193066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.193648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.195187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.197839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.202000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 464/907] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:23.207871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:23.208771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:23.210250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:23.210675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.212103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 444/908] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:23.213458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.224799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.322391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c6698-f503-4e0f-b650-1206d7f272ee None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.353129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90899-ec70-45f0-85bb-8902a4b86290 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.361839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.362773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.363277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.363811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.364912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] RBAC: Adding query filter params (interface=public, service_id=57e9c8494678484a9d9dcb37f355ab85, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.365501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.367086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.377778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 465/909] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/endpoints?service_id=57e9c8494678484a9d9dcb37f355ab85&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.378998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.393144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.393851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.394137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.394545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.395482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.395798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.397648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.415501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 445/910] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:23.421413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.462951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.505660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.512275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e00ba3-2995-4ad6-842a-66e2112e099f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.542312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9c280-1dd2-426c-9444-a2a4bda8a718 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.549264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.550077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.550365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.550643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.551328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.553650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55d645-52bd-46aa-822d-a80811bdf8e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 466/911] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632df49-b1f9-4fbe-882a-93618fc06021 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632df49-b1f9-4fbe-882a-93618fc06021 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632df49-b1f9-4fbe-882a-93618fc06021 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 467/912] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.569493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.570067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.590858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:23.592401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:23.602604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.603282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.603570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.603849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.605213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] RBAC: Adding query filter params (role.id=92650894c3fb4749b87bd354b92e58de, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=b1f2a519e5cc42a993a6007174343edf) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:23.605563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.607852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.616183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 446/913] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/role_assignments?user.id=b1f2a519e5cc42a993a6007174343edf&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=92650894c3fb4749b87bd354b92e58de => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:23.682450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 468/914] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:23.707050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.827809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8602-09c8-46d6-bab5-ca25b4d5da9c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:23.857148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.861973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:23.862773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:23.863308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:23.863581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:23.864234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:23.866301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:23.878497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:23.879511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:23.880347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 447/915] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:23.886180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:23.950278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:23.993397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f1f5c-8755-4069-a3d2-07a906e34a48 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:24.028612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:24.029281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:24.029699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:24.030119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:24.030841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:24.031152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:24.032526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:24.037836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 469/916] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:24.043271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:24.153110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3a604-7577-40eb-97df-b1495a3d6eb5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:24.188571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:24.189225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:24.189530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:24.189806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:24.190507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:24.190809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:24.192225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a8c1e-5bfb-4122-b348-7a6b329840b7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:24.198215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 448/917] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Jul 12 14:20:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:24.203504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:24.307447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a8690-1099-4786-9c4a-eab76242a5a4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:24.330975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:24.343304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:24.344014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:24.344299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:24.344574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:24.345432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:24.346810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b2426-cb4b-4d05-8a5e-95cfa7277ccd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:24.380418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 470/918] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:24.721749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84aa-5988-46fe-94df-50c9efe7a4fd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:24.722027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84aa-5988-46fe-94df-50c9efe7a4fd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:24.722304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e84aa-5988-46fe-94df-50c9efe7a4fd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:24.723842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 449/919] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:24.729366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:24.729838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:24.729921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:24.747705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:24.748913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:24.830833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 471/920] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:24.854172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e84aa-5988-46fe-94df-50c9efe7a4fd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:24.916515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:24.964428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e84aa-5988-46fe-94df-50c9efe7a4fd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:24.999677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.000785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.001252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.001734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.002567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.004098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:25.016969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:25.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:25.018904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:25.021276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 450/921] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:25.033396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.130745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d57e-e5d3-42e4-9e72-40666dca87d4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.133701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.169378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.170047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.170334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.170614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.171328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:25.171637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.173017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.178630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 472/922] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.184008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.294502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3ec9-a5f8-4016-970f-4bc83e56a78d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.319737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.331229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.332095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.332381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.332665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.333405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:25.333719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.335640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.340995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 451/923] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.350514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.454096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0a771-615f-44bc-b29e-f89d41764524 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.489074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.489769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.490052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.490840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.491704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] RBAC: Adding query filter params (interface=public, service_id=f492d2a6177c427eaf3bbab0e752139e, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:25.492023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.493417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92a836-eb2c-48b1-a213-1ef84f494821 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.498981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 473/924] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/endpoints?service_id=f492d2a6177c427eaf3bbab0e752139e&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:25.504272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.608798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259a828-edce-4de8-bc11-1cdc31bb66ae None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:25.650215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:25.651176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.651536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.651822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.652515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:25.653926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d84920-eed3-4f99-8c3d-8aa0709080a1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:25.660661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 452/925] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:20:25.741776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff940b-e597-46a4-867c-339ad8836c9d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.742445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff940b-e597-46a4-867c-339ad8836c9d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.742445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff940b-e597-46a4-867c-339ad8836c9d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.744026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 474/926] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:25.749917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.750210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.750475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.768520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32dfb-9163-485a-bc11-563f0f4f5142 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.769121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:25.769435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32dfb-9163-485a-bc11-563f0f4f5142 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.769938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32dfb-9163-485a-bc11-563f0f4f5142 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.770360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:25.772167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 475/927] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:25.778110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:25.778744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:25.779280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:25.796902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:25.798240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:25.830953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.835149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:25.860108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 453/928] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:25.884454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:25.888858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 476/929] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:25.912253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.004811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf440c9-bb35-42f1-9a15-c158f91abffd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.028086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb9a2f-ac2a-4af4-9a67-0c92243098dc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.048163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.049683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.050148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.050620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.064080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:26.070900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.071962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.072540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:26.073032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:26.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:26.079899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:26.080562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.080633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 454/930] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:26.080710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.080710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.082304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:26.096018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:26.096998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:26.102894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 477/931] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Jul 12 14:20:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:26.105516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.140719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.144630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.230460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25f4a-2b28-48a6-880f-593fe2cabc13 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.231270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05174a1-9488-49bc-a436-6bdd0d1dd30e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.269826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.270972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.271267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.272132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.273161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.273518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.275183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.280538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.281546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.282045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.282533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.283438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.283938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.285368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 455/932] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:26.290512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.292220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.300909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 478/933] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.302430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.399006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.410705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db066-f6b0-48c0-bb32-a9c915fe8bfb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.440646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb8a3c-8ff8-42c1-a51b-3f081617e06f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.445315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.447338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.448080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.448550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.448896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.449736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.450020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.451388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.461980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 479/934] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.463390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.489956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.491803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.492321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.493062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.508640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.511115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:26.530590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:26.531532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:26.532482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:26.532482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:26.532482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:26.532482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:26.536438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 456/935] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:26.538225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.595440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2141b1-d558-4ea4-be6b-54bf154edae0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.632747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.633455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.633743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.634020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.634755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.636130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6bbf4-032c-4e74-b25e-87a5938feb92 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.653157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79dc42-37d8-40a4-a7de-2edf25dd85bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.656084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 480/936] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:26.657125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b724-7d12-499d-ae46-296d8cec4603 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.657546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b724-7d12-499d-ae46-296d8cec4603 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.657808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0b724-7d12-499d-ae46-296d8cec4603 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.659138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 481/937] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:26.665286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.665629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.665917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.691195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:26.692520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:26.697526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.732531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.733836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.734543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.735442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.736642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:26.737115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.738765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.757908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 457/938] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:26.763486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.788881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 482/939] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:26.796445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:26.813596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.898002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b33e10-1aed-4d68-b57c-c978815b0de1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.934778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbd0ab-fa90-48c4-a4d1-9d102f7d1842 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:26.946449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.947858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.948326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.949067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.959834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.961928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.977371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:26.978271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:26.978758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:26.979198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:26.979723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:26.980851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:26.982073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:26.983263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:26.984937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:26.986560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING keystone.server.flask.common [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 14:20:26.987704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 458/940] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:26.988951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:26.999426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.010645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:20:27.010645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 14:20:27.010645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [parameters: {'id': '0a716d0ddef24024bdfe2be6e677c4b1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 14:20:27.010645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a716d0ddef24024bdfe2be6e677c4b1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a716d0ddef24024bdfe2be6e677c4b1. Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:27.034187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 14:20:27.035040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() Jul 12 14:20:27.035993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:20:27.036871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:20:27.038386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:27.039159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'id': '0a716d0ddef24024bdfe2be6e677c4b1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.042700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:27.044597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a716d0ddef24024bdfe2be6e677c4b1. Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.047220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 483/941] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Jul 12 14:20:26 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:27.048658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.080281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.115385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d6aab-b929-4b8c-8d4d-d94390ec4f1d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.155139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.155811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.156099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.156374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.157210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:27.157561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.159120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.184226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 459/942] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:27.185760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.186788 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892427b-7431-461a-982b-b063b3b04534 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.234729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.236841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.237380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.238111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.248198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.251131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:27.273179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:27.274323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:27.275402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.281594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 484/943] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:27.283895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.296461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6278a6-4410-4e60-b8bb-72d951b19502 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.326523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.332984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.333931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.335059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.335944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.346790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.348346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.350768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:27.367532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:27.368642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:27.369816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:27.369816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:27.369816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:27.369816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:27.375663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 460/944] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:27.377372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.405246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8ff6-29d5-4052-9254-0d913d7ac221 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.447726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.448698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.449226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.449685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.450693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:27.451153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.452724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.481793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 485/945] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:27.485521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f41bb-844f-4972-8979-d8314e7f9a63 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.526787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.527828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.528254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.528623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.529531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:27.530046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.531562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.549498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 461/946] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:27.554609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.667219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ed57-5c38-452d-82aa-67e81327382d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.687254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.707626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.708640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.709654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.710078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.717586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.719219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-abedb104-3105-4289-a542-ce83c8c56211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:27.734296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:27.735381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:27.736462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:27.741373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 486/947] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:27.744505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.784556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49db4-bd2b-4f57-befa-47669308187f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.784885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49db4-bd2b-4f57-befa-47669308187f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.785106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49db4-bd2b-4f57-befa-47669308187f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.786782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 487/948] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:27.794659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.795649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.796177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.798376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:27.814505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:27.816350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:27.871330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a067e8c-85a7-48b3-9451-4668d96a0e94 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:27.889136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 488/949] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:27.915153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:27.916998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:27.918252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:27.918860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:27.919344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:27.920355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:27.920895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:27.922697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:27.936951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 462/950] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:27.944550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.046418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa1f1f-60c8-4748-ab0f-59a67e9f638b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.055993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.070638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e91a-56d2-4dc9-b120-3148264c7e25 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.073667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.091679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.092892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.093183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.093496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.094159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.095982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.110560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.111577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:28.111742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:28.112875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:28.113775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 489/951] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:28.114535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.114676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/b1f2a519e5cc42a993a6007174343edf/roles/92650894c3fb4749b87bd354b92e58de` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.120924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=b1f2a519e5cc42a993a6007174343edf, role_id=92650894c3fb4749b87bd354b92e58de)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.122245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.122381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.141408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 463/952] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:27 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/b1f2a519e5cc42a993a6007174343edf/roles/92650894c3fb4749b87bd354b92e58de => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:28.243633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a71e-ac8e-4853-a297-259e7a332d26 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.282081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.283133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.283645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.284127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.285117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:28.285670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.287398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.293901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 490/953] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:28.303398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.419826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00672-2bb4-4a61-8bd2-80b90b78d47d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.457112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.458347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.458841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.459864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.460769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:28.461364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.462988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ed29f-ff85-4f6c-a10a-fe04ba86033f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.469136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 464/954] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:28.477740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.514497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:28.586010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6723-5143-4966-a0be-85d1e91b1d82 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:28.621688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:28.622351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.622632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.622907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.623641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:28.625155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc2e1-e81d-475f-8689-d218dc80a5d3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:28.653518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 491/955] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:28.803668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ec35f-926f-4707-90af-6102e95ac84b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.803993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ec35f-926f-4707-90af-6102e95ac84b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.804240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ec35f-926f-4707-90af-6102e95ac84b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.805919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 465/956] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:28.811121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:28.811572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:28.811695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:28.829016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:28.830308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:28.905672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 492/957] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:28.928096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ec35f-926f-4707-90af-6102e95ac84b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:28.957801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.035903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ec35f-926f-4707-90af-6102e95ac84b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.071601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.072469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.072753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.073027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.073705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.075189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:29.087683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:29.088771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:29.089746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 466/958] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:29.097493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.115884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.200907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42630a3-e197-4404-a6b1-9843d02e846d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.236174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.236848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.237126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.237425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.238134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:29.238435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.239923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c195ae2-0f0a-40a2-88f5-8a8505f7cb64 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.245755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 493/959] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:29.251252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.347712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.358725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb253-9942-4daf-81ea-a7fc9404243d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.396418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.397210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.397528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.397810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.398587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:29.398900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.400262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12a3d-fe53-45aa-9a82-58c12bb8f570 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.406127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 467/960] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:29.667906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf7bd5-8d3f-475b-b00b-0181734f9360 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.668320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf7bd5-8d3f-475b-b00b-0181734f9360 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.668512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf7bd5-8d3f-475b-b00b-0181734f9360 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.670131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 494/961] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:29.676524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.677220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.677739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.697791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:29.698988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:29.773182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 468/962] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:29.797712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf7bd5-8d3f-475b-b00b-0181734f9360 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:29.889246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:29.917548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf7bd5-8d3f-475b-b00b-0181734f9360 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:29.954678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:29.955721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:29.956211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:29.956738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:29.957693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:29.959921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:29.979711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.sql.core [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:20:29.979711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 14:20:29.979711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [parameters: {'user_id': 'fa805c82a23249c5b18fe3bfcbac7d5c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 14:20:29.979711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:29.987169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:29.988032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application yield resource Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application next(self.gen) Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise self.value Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application session.commit() Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 14:20:29.989002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.session.flush() Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application compat.raise_( Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 14:20:29.989955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 14:20:29.991419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application conn.query(q) Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application result.read() Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 14:20:29.992298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa805c82a23249c5b18fe3bfcbac7d5c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:29.993141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 14:20:29.995972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:29.998279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 495/963] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Jul 12 14:20:29 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:30.007113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.090211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.107644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b084f5-9160-4fba-b6c7-49cd93b856c7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.144027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61012589-56b0-4651-8073-4572cf436477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.144828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012589-56b0-4651-8073-4572cf436477 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.145113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012589-56b0-4651-8073-4572cf436477 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.145414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012589-56b0-4651-8073-4572cf436477 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.152840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012589-56b0-4651-8073-4572cf436477 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.154306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012589-56b0-4651-8073-4572cf436477 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-61012589-56b0-4651-8073-4572cf436477 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:30.169739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:30.171644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:30.173655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 469/964] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:30.177464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.284526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3385f6-6b55-400f-8163-33194525bfa0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.321566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.322375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.322660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.322936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.323798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:30.324109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.325834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.339463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5013e-6e57-4a3a-a6e6-5d8d521b6a12 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.339782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5013e-6e57-4a3a-a6e6-5d8d521b6a12 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.340023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5013e-6e57-4a3a-a6e6-5d8d521b6a12 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.341675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 470/965] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:30.349193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.350285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.350767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.365477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.368384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:30.370453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:30.378253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 496/966] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:30.456659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 471/967] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:30.480867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.595979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc40e3-1d69-4700-9713-12ba987b44bd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.633099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.634004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.634295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.634548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.645883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d855ab29-8080-4b8a-a101-b38cc7a50f1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:30.654688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:30.655631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:30.656486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 497/968] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:30.661926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.741144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.768744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de439285-5f3f-40d6-b2cc-6fdeae9b1a30 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.804213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.804881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.805162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.805465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.805696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454f390-067b-4096-943c-02fe2a6b482e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.805989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454f390-067b-4096-943c-02fe2a6b482e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.806207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:30.806307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454f390-067b-4096-943c-02fe2a6b482e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.806515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.807755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 498/969] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:30.807995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:30.812462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.812751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.813034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.817629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 472/970] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:30.822968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.831681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:30.833207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:30.894361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.909490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 499/971] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:30.929810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab9583-2cf6-4854-8f2e-99f4dd86a325 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:30.930787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:30.967042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:30.968131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:30.968599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:30.969210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:30.972022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:30.983119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:30.984571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:31.003303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:31.004237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:31.010536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:31.010536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:31.010536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:31.010536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:31.010536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 473/972] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:31.010886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.036519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2471-8c25-441e-818c-ee4a6825839c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.073305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.074139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.074421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.074694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.075347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.076706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:31.091643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:31.092655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:31.093510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 500/973] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:31.098651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.120650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e838ef-9177-4daf-9a4b-e248360c90fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.154745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.155317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.155603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.155874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.156593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.156901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.158291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.165757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.172864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 474/974] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:31.179183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.224716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a854-0d78-4564-b745-f69e26c32a0e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.230317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.262662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.263325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.263608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.263881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.264584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.264893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.266279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.271037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 501/975] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.277121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.292689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299ade1-dadf-4eea-9b12-7694b68dfa57 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.327317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.328129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.328412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.328689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.336344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.337795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:31.352655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:31.353680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:31.354875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 475/976] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:31.359596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.385104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195f74-af06-41bd-a335-c6b503996498 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.408335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.431459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.432136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.432419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.432697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.433437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.433749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.435106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.440351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 502/977] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.445683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.476954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fc7f1-e90e-4246-8eaa-91c4e0690eb2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.487308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.514277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.515199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.515199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.515487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.516324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.516637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.518086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021de23c-88db-4fac-9970-1e0b2065f7ba None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.536690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 476/978] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:31.538701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ab38-d29c-4338-a948-fcb13a227961 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.538993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ab38-d29c-4338-a948-fcb13a227961 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.539280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ab38-d29c-4338-a948-fcb13a227961 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.540515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 477/979] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:31.542882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454ab38-d29c-4338-a948-fcb13a227961 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.550720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbb18-181c-4a09-bee3-e3e15a06ea1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.584838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.585689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.586034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.586313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.587215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] RBAC: Adding query filter params (interface=admin, service_id=f492d2a6177c427eaf3bbab0e752139e, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:31.587531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.588970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68541d4-e9b2-446e-b85f-b346a39ae40e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.594087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 503/980] 10.222.0.25 () {56 vars in 1249 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/endpoints?service_id=f492d2a6177c427eaf3bbab0e752139e&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:31.595243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.595525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.595799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.612285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:31.613479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:31.645055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454ab38-d29c-4338-a948-fcb13a227961 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.656373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.679917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.680500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.680783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.681057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.682026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 504/981] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:31.682773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.721566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.758377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.759821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.778442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712f22-8199-4f94-996a-38428c0a54a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.811591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.812355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.812532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.812811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.813523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.814880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:31.821262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 505/982] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/services => generated 1497 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:20:31.822040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.871320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.884889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 478/983] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:20:31.886253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:31.942315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40acf28-73ef-4635-ac62-499b5619a46d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:31.968565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:31.985668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:31.986482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:31.986767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:31.987041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:31.987764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:31.989118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.004547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 506/984] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:31 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:32.007556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b3ac2-6587-4c54-ab85-ecc9ad950c77 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.055081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.055901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.056174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.056446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.068687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.070209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:32.090269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:32.091231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:32.092127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:32.092127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:32.092127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:32.092127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:32.092127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 479/985] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:32.097382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.206230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.210013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427aaee-50dd-438d-a014-d2643b1bdbd9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.246435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.247099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.247381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.247651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.248364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:32.248666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.250045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.266493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 507/986] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:32.267800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.372441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88f0a4-ec2e-4e67-8741-89859d4e3c34 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.387722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:32.416914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.417731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.418017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.418289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.425501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.427201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:32.441265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:32.442206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:32.443175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:32.445399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 480/987] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:32.448769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.553947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3861f-5584-4de2-8efd-f8cca76a426c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.587797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.588459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.588741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.589014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.589821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:32.590130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.591544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea03a5d-7f67-4100-858f-bddba70d6e0e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.603408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 508/988] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:32.609073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:32.716146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318066-a2e1-4039-a84e-c555806bf30d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:32.752473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:32.753121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:32.753478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:32.753758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:32.754574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] RBAC: Adding query filter params (role.id=92650894c3fb4749b87bd354b92e58de, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=b1f2a519e5cc42a993a6007174343edf) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:32.754891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:32.756340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96e66e-94b7-48c0-aeea-3dd5294c673e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:32.769372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 481/989] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:32 2022] GET /identity/v3/role_assignments?user.id=b1f2a519e5cc42a993a6007174343edf&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=92650894c3fb4749b87bd354b92e58de => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:33.848117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8d64-600e-460c-9087-fa6969451940 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.848618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8d64-600e-460c-9087-fa6969451940 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.848723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8d64-600e-460c-9087-fa6969451940 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.850296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 509/990] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:33.855442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146d2e-363c-4288-8ba5-d899c9b4da1f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:33.855846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146d2e-363c-4288-8ba5-d899c9b4da1f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:33.856013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146d2e-363c-4288-8ba5-d899c9b4da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:33.873280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-83146d2e-363c-4288-8ba5-d899c9b4da1f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:33.874528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-83146d2e-363c-4288-8ba5-d899c9b4da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:33.908588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:33.948796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 482/991] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:33.971342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e8d64-600e-460c-9087-fa6969451940 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:33.993515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.049607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00003f-e08d-408a-9396-a0317ddf396e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.049921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00003f-e08d-408a-9396-a0317ddf396e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.050154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00003f-e08d-408a-9396-a0317ddf396e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.051640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 483/992] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:34.056655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.056958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.057238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.074451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:34.075620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:34.076578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e8d64-600e-460c-9087-fa6969451940 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.116355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.117084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.117383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.117659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.118303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.119731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:34.133521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:34.134501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:34.135346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 510/993] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Jul 12 14:20:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:34.140385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.148010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 484/994] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:34.170933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.227921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.247954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f867f-501c-4a90-9255-9311a71beef1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.269689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.275965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff121-db03-4e9c-8acd-45a9311c5357 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.289317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.290262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.290692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.291120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.292102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.292569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.294161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.299721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 511/995] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:34.305695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.314414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.315211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.315492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.315764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.316409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.317796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:34.329618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:34.330780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:34.331631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 485/996] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:34.338359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.423889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003980b-b0a6-4000-b375-12e8b673e8f0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.453511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d5bb-99b1-4140-8813-73e2b0270910 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.458699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.459961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.460316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.460664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.461486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.461860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.463231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.469365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 512/997] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:34.473810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.494693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.495319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.495437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.495632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.495885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.496587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.497188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.498287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.504162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 486/998] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:34.510420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.539570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.587244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e8c8e-db72-47ca-a07d-b4b29aabd6f1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.621968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293e446-6824-4593-819f-475d86eb739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.622634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293e446-6824-4593-819f-475d86eb739c None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.622921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293e446-6824-4593-819f-475d86eb739c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.623198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293e446-6824-4593-819f-475d86eb739c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.623957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293e446-6824-4593-819f-475d86eb739c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.624416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d491e-ad03-42a5-8866-18adf4d87eb9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.625357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293e446-6824-4593-819f-475d86eb739c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.642961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 513/999] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:34.671546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.672141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.672424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.672702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.673515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.673830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.675232 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ea79-42cc-49d8-a1bb-56f852c99062 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.686936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 487/1000] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:34.688497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293e446-6824-4593-819f-475d86eb739c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.767920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.805968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293e446-6824-4593-819f-475d86eb739c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:34.817952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bbdb1-444f-40c8-a2b6-a479ed716c69 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.819025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bbdb1-444f-40c8-a2b6-a479ed716c69 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.819301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bbdb1-444f-40c8-a2b6-a479ed716c69 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.820658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 488/1001] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:34.826417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.826703 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.826983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.844361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:34.845480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:34.845764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:34.846189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:34.846677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:34.846951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:34.847908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] RBAC: Adding query filter params (interface=public, service_id=206fd2d66fc54585b58bb3919c20c4a2, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:34.848423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:34.850489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:34.856237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 514/1002] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/endpoints?service_id=206fd2d66fc54585b58bb3919c20c4a2&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:34.861490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.920898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 489/1003] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:34.944411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:34.963123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:34.973130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cfc10-8b90-4ecf-839e-b3a895eae1a9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.010153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.011103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.011538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.011963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.012798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.014345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.016479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:35.034470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 515/1004] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:20:35.063298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc6425-0ca7-43ad-8c56-c26af44e1ceb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.101737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.102548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.102828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.103100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.103868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.105318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:35.118077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:35.119142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:35.125411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 490/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 14:20:34 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:35.126738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:35.240784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9782e4e-3438-4c2c-8cd9-422e4da3485c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.275224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.275891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.276173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.276447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.277150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:35.277485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.278839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af65b5-8baa-4d53-bb9f-23f2ac091ba3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.283983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 516/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:35.290372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:35.375354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:35.396744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463f7c-c340-4aa7-a921-252c6a04481e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:35.437724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:35.438391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:35.438674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:35.438949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:35.439652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] RBAC: Adding query filter params (type=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:35.439960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:35.441312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095e2d5-8537-4df8-ade9-81900e00006b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:35.453378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 491/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:36.711123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531252-f02b-4b21-a68e-698153cc0d98 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.711605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531252-f02b-4b21-a68e-698153cc0d98 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.711605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c531252-f02b-4b21-a68e-698153cc0d98 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.713266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 517/1008] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:36.718591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.719003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.719142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.739756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:36.741520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:36.814739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 492/1009] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:36.838555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531252-f02b-4b21-a68e-698153cc0d98 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:36.904801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:36.945872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c531252-f02b-4b21-a68e-698153cc0d98 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:36.980448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:36.981159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:36.981537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:36.981813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:36.982974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:36.984479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d0a9e-72c2-4fc9-bc30-c3693816ce39 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:36.986298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 518/1010] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:36.992525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.069247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.083942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8acf-a71f-40aa-a8c0-67a4af83955d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.084726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8acf-a71f-40aa-a8c0-67a4af83955d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.084996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e8acf-a71f-40aa-a8c0-67a4af83955d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.086396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 519/1011] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:37.091326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.091612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.091892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.098565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869c481-0655-4c23-8cb3-44c4312f4027 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.110038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:37.111234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:37.137314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.137957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.138237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.138509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.139222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.140637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.168718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 493/1012] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:36 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:37.185373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 520/1013] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:37.207383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.322371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea8f7-55e7-485a-92c0-482d19b55b68 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.352063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.362267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.363062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.363347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.363626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.364279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.365727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d8dad7ae-b0c7-4f6b-9d4b-85b6f2533195 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:37.377860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:37.378871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:37.379763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 494/1014] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:37.385275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.456369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.494826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b74d-ded2-4bce-b45a-063621ca5d57 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.495151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b74d-ded2-4bce-b45a-063621ca5d57 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.495419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b74d-ded2-4bce-b45a-063621ca5d57 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.496431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b7e5e-5a57-428d-89cb-f7427e4a70ac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.496897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 495/1015] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:37.501634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.501927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.502207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.520105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:37.521359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:37.531124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.531792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.532071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.532345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.533047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:37.533375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.534728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.539742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 521/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:37.545149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.602963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 496/1017] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:37.626016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.650768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52253-dfe3-4447-ab1b-d8b707d93e4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.679774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.690593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.691305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.691594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.691970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.692577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:37.692891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.694818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.704382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 522/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Jul 12 14:20:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:37.705622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:37.736394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ca7c-3ac3-42c2-937f-3d21ff535fd0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.759962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:37.775144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.775837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.776468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.776468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.777115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.778490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f8096-bb8c-4d3d-99a8-6701ad953f70 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.805363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 497/1019] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:20:37 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:37.812603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d20d71-1faa-4953-b848-9a8dd52d4b7c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:37.859627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:37.860212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:37.860501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:37.860775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:37.861542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:37.862945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c4bf0-9ed3-4d74-9493-d7e389cd1859 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:37.883985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 523/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:39.236535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebe376-34e2-4d51-9b1e-353974ed9528 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.237057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebe376-34e2-4d51-9b1e-353974ed9528 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.237172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebe376-34e2-4d51-9b1e-353974ed9528 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.238509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 498/1021] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:39.243661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.244037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.244225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.261121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:39.262323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:39.298676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:39.338589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 524/1022] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:39.362738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebe376-34e2-4d51-9b1e-353974ed9528 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.470504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebe376-34e2-4d51-9b1e-353974ed9528 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.504387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:39.505088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.505391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.505702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.506859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:39.508278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13917633-0e53-4534-8fd5-44130c618970 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:39.510353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 499/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:39.515902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:39.614548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fa4bb-937a-4b46-a531-9a7603b26250 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:39.649318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:39.650012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.650294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.650567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.651282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:39.652795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb67a2-15c2-49a3-a382-164bd7d80609 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:39.673652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 525/1024] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:39 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:39.844669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6c2c3-1589-4d42-bc25-c2dbbcadf1fa None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.845020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6c2c3-1589-4d42-bc25-c2dbbcadf1fa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.845238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6c2c3-1589-4d42-bc25-c2dbbcadf1fa None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.846816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 500/1025] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:39.852751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb204b3-1664-408f-a550-5a62a8f48fec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:39.853302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb204b3-1664-408f-a550-5a62a8f48fec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:39.853828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb204b3-1664-408f-a550-5a62a8f48fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:39.871346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0eb204b3-1664-408f-a550-5a62a8f48fec None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:39.872803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0eb204b3-1664-408f-a550-5a62a8f48fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:39.892252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:39.944810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 526/1026] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:39.968180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6c2c3-1589-4d42-bc25-c2dbbcadf1fa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.009798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec89de-0ff2-4b94-bcc7-1bc3f88522bf None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.010183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec89de-0ff2-4b94-bcc7-1bc3f88522bf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.010473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec89de-0ff2-4b94-bcc7-1bc3f88522bf None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.011994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 527/1027] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:40.017192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.017512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.017790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.025368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.035552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:40.036754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:40.089515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6c2c3-1589-4d42-bc25-c2dbbcadf1fa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.109026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 528/1028] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:40.129676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.130583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.130875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.131187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.131926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.133283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.133509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:40.152139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:40.153177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:40.154614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 501/1029] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 14:20:39 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:40.159705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.231225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.249164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120fdb7-969e-4616-bb93-f8249f83935a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.268824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea48f28-b56f-41a3-b7b7-1e6a31eb2433 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.284419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.285670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.286843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.287938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.288820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.299057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:40.307592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:40.308712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:40.309716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 529/1030] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:40.311726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.312379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.312664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.312937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.313751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.314287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.315119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.315408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.321816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 502/1031] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.327280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.425173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0b4c-1574-46a8-87cf-c5ca40eae98f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.441959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489d63-2e84-4ccb-bda5-4880fc7cacc0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.461042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.461737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.462018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.462293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.463041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.463458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.464851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.473674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 530/1032] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.479159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.484439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.485303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.485786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.486213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.487089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] RBAC: Adding query filter params (type=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.487553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.487906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.489592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.502027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 503/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.503260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.555086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.583890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f569-881b-4b07-9740-be38f78b7ffc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.617820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effa93-afbf-4646-a14a-c5d4e646898c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.621513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.622334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.622616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.622891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.634917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.636346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:40.652927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:40.654020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:40.655079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:40.655079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:40.655079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:40.655079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:40.655079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 531/1034] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:40.657535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.658364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.658814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.659252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.659777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.660363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] RBAC: Adding query filter params (interface=public, service_id=8e73ec90856e49adb5ed472711636235, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.660837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.663037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.668601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 504/1035] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/endpoints?service_id=8e73ec90856e49adb5ed472711636235&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.673865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.722945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.771085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b296114-a7ce-472b-b9a3-6e197b1c1650 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.787734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2086c4-6634-43b8-9839-c8fd71546188 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.790930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.807230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.807896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.808176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.808447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.809163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:40.809514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.810869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.827879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 532/1036] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:40.828968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:40.835296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.835968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.836253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.836531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.837208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.839072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:40.853386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 505/1037] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:20:40.945949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52367f3-7dfc-4661-b739-64d95b5ac0df None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:40.964899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:40.983474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:40.984285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:40.984563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:40.984835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:40.993224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:40.994792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:41.011408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:41.012381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:41.013354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 533/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:20:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:41.018573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.119226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9dae0-6c70-4f35-9174-b083aefd6a96 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:41.154261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:41.154920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.155200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.155476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.156296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:41.156603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:41.158034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.175634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 506/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:41.182203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.281828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b5c84-f48f-4a91-a82c-4ad985b63523 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:41.315577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:41.316219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.316497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.316775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.364749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:41.388185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:41.389652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.493617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 534/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:41.499574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.527412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:41.600986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f48d3-62d0-4d8e-a3fa-17fadfe85369 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:41.635112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:41.635902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.636184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.636459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.646655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:41.648075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3a50ff97-0db0-4a58-971f-d5eafdd1aed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:41.665711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:41.666690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:41.667626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:41.667626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:41.667626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:41.667626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:41.669992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 507/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:41.674596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.751599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:41.774494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584059a8-0cad-4411-a923-5e6c09608095 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:41.779486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470a49-1954-4147-a0d2-dccde8a4d454 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.779784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470a49-1954-4147-a0d2-dccde8a4d454 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.780068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20470a49-1954-4147-a0d2-dccde8a4d454 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.781598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 508/1042] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:41.787418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.787713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.787991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.808294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:41.809509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:41.810595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:41.811265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.811563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.811830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.812555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:41.812864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:41.814284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.829985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 535/1043] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:41.832101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.888773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 509/1044] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:41.911930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:41.924812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:41.934799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b0eeb-9e07-4af5-9a80-0784eba0e1cc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:41.973448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:41.974468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:41.975063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:41.975186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:41.983455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:41.985061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:41.987386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:42.004378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:42.005658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:42.006595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 536/1045] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:42.011583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.023930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba559b8-93a0-4418-9321-c1860ec83d86 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.062010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.062805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.063195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.063474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.074352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:42.081990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:42.082948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:42.084345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 510/1046] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:42.089704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.119837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4a00e-402b-4f88-9329-369135681701 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.158717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.159560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.159959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.160360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.160536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.161484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:42.162373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.164737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.182798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 537/1047] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:42.183590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.202052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a894d24-1dd6-4310-a573-6b7c0515165b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.238505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.239276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.239559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.239837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.240600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:42.240906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.241305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.243529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.253630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 511/1048] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:42.259847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.287763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf493d-8e6c-404e-9203-d578095effbd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.322586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.323236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.323528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.323808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.324641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] RBAC: Adding query filter params (role.id=92650894c3fb4749b87bd354b92e58de, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:42.324956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.326434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.341367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 538/1049] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=92650894c3fb4749b87bd354b92e58de => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:42.377467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e62e6-9fd0-4035-9f31-0ce67375b612 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.415294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.422643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.423530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.423818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.424100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] PATH_INFO: `/v3/users/f61c2ef3c9ea40e599da260eb78e43bf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.440532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] RBAC: Authorizing `identity:get_user(user_id=f61c2ef3c9ea40e599da260eb78e43bf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.442038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.448743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 512/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/users/f61c2ef3c9ea40e599da260eb78e43bf => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:42.449799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.555755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d0e7-f1a4-4184-9a32-5f34f6018444 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.577842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:42.590843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.591498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.591781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.592058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.660158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.661620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.765309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 539/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:20:42.766474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.869470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a01e-b386-4f3f-a030-1cb7e4e23164 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:42.913147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:42.914039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.914322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.914594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.915766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:42.917184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318038a8-1b27-4684-8995-71cdbee3ae9e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:42.919054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 513/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:42.925010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:42.928305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7c25c-1d32-49d0-a34b-e39f0860cd8b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.928613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7c25c-1d32-49d0-a34b-e39f0860cd8b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.928894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7c25c-1d32-49d0-a34b-e39f0860cd8b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.931449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 514/1053] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:42.935440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:42.935818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:42.936107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:42.953295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:42.954579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:42.960498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.010295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.030412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 515/1054] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:43.047000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3decbe7-5888-453d-b58f-dda42ea4da96 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.053117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.085900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.086553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.086841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.087133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.088010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=963a41c11d45405f80b4668c2d931163, user.id=f61c2ef3c9ea40e599da260eb78e43bf) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.088322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.089787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.097227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 540/1055] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:42 2022] GET /identity/v3/role_assignments?user.id=f61c2ef3c9ea40e599da260eb78e43bf&scope.project.id=963a41c11d45405f80b4668c2d931163&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.173845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6bda1-84b4-4136-8c2b-048249978c23 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.210710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.211516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.211798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.212074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.212735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.214155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:43.226647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:43.227885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 14:20:43.228730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:43.233378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 516/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:43.234160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.327951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.345651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea6a98-d88a-4158-86be-86bb0f59bbdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.381699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.382369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.382655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.382933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.383639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] RBAC: Adding query filter params (name=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.383947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.385314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.392120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 541/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.398186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.461040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:43.514699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f1950-ee59-4df7-80f6-6ed853e82bdc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.549610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.550279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.550567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.550845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.551549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:43.551857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.553597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abdde1-ccd1-499a-a5df-034d7d555de0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.563235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 517/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:43.564429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:43.667246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeef19-8f90-4292-be92-c7691a6e9272 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:43.700761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:43.701447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:43.701735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:43.702014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:43.702761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:43.704116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c17e8-4256-4946-8dc4-93bc9033bacb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:43.722245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 542/1059] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:20:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:20:44.385681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c730bc-77ea-4fd0-b66d-aa9c3f7b9c81 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.386204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c730bc-77ea-4fd0-b66d-aa9c3f7b9c81 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.386283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c730bc-77ea-4fd0-b66d-aa9c3f7b9c81 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.387622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 518/1060] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:44.392710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.393095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.393273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.406948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.417793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:44.419005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:44.492513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 543/1061] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:44.514523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c730bc-77ea-4fd0-b66d-aa9c3f7b9c81 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.615942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c730bc-77ea-4fd0-b66d-aa9c3f7b9c81 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.631240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.652835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.653606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.653893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.654224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.664535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:44.672867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:44.673989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:44.674862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:44.677501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 519/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:44.680657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.780355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642ee3e-066e-4c11-89c2-1e49c8faa905 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.810540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:44.816584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.817260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.817573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.817851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.818638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:44.818938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.820303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:44.829520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 544/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:44.835165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:44.941235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713f8d8-c882-46ab-990b-e657f7f7062c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:44.975634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:44.976408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:44.976688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:44.976956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:44.987379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:44.988800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:45.007011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.007902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.013749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:45.013749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:45.013749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:45.013749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:45.013749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 520/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:45.014119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.115653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407f729-952b-4a7b-b8c7-4240ec8172e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.149083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.150054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.150519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.150998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.151950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.152435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.153996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0725b1a-ee31-41f0-b6df-968a3f73807f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.171556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 545/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.172625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.189719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a846-3ed5-44e3-a1cc-da09697335ff None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.190291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a846-3ed5-44e3-a1cc-da09697335ff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.190291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2a846-3ed5-44e3-a1cc-da09697335ff None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.191794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 546/1066] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:45.196785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.196921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.197213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.204056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.214303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:45.215476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:45.266751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.284191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907d44-b24a-4319-ae4b-c60d72717c0e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.285263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 547/1067] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:45.307969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.321644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.322448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.322838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.323013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.331679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.333103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:45.349123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.350629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:45.351579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 521/1068] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:45.356427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.420121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bc72b-e3bd-49c0-b379-136f43b1a0a5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.453104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.460129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.460928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.461211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.461522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.470720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c670144-37cd-4747-9eaf-a9d1f9c64c89 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.474429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:45.483545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.484485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:45.485321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:45.486652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 548/1069] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:45.491772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.511121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.511787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.512069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.512345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.513176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.513520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.514943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.539573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 522/1070] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.540714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.541125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.603923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59a84d-180c-4f63-8966-3d24cb8546db None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.648168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.649133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de030da0-601c-4a0c-b3b4-93b752f07a29 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.649260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.649722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.650157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.652034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.652517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.654073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.667800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 549/1071] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.669076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.691708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.692489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.692773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.693109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.704085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.705485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.708322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:45.725972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:45.726944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:45.727887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:45.727887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:45.727887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:45.727887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:45.727887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 523/1072] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:45.732813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.775798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d09a07-1d36-4329-8187-c0be58971c69 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.783469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.811016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.811906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.812184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.812753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] PATH_INFO: `/v3/users/f61c2ef3c9ea40e599da260eb78e43bf` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.813657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] RBAC: Authorizing `identity:get_user(user_id=f61c2ef3c9ea40e599da260eb78e43bf)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.815066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c07c46-9e37-41dc-92c3-21f52a3ad2df None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.816683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 550/1073] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/users/f61c2ef3c9ea40e599da260eb78e43bf => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.817854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab067-e557-46f4-ad34-de85b8b0a1f9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.818863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab067-e557-46f4-ad34-de85b8b0a1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.819287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab067-e557-46f4-ad34-de85b8b0a1f9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.820823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 551/1074] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:45.823106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ab067-e557-46f4-ad34-de85b8b0a1f9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.840830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308bbd-39e7-4d22-9e01-271356f46c9c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.874801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.875456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.875743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.876023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.876746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:45.877055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.878452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92f43c-4417-4c3e-b682-2e01d585844c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.890010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 524/1075] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.891261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.891551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.891834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.907374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:45.908569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:45.926432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ab067-e557-46f4-ad34-de85b8b0a1f9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:45.959894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:45.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:45.961323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:45.961632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:45.961941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:45.963133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:45.964555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:45.971067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 552/1076] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:45.971067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:45.978936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 525/1077] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:45.979595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.013868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.069679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293ce2-e7e0-4247-8264-c1858e986a65 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.085585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995531-72c4-4daf-b696-a535e3ff5e88 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.103188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.103980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.104265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.104544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.111335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.112782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.119965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.120878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.121159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.121469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163/users/f61c2ef3c9ea40e599da260eb78e43bf/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.123435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] RBAC: Authorizing `identity:create_grant(project_id=963a41c11d45405f80b4668c2d931163, user_id=f61c2ef3c9ea40e599da260eb78e43bf, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.124866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:46.127052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:46.128187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 553/1078] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:46.129371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.143728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 526/1079] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:45 2022] PUT /identity/v3/projects/963a41c11d45405f80b4668c2d931163/users/f61c2ef3c9ea40e599da260eb78e43bf/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:46.144518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.199475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.247387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71582ef1-ee00-4a55-b0c3-1dee57b1bff3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.249081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.267099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5de98b-ea9f-49b6-99cf-45bda4240deb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.286019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.286975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.287265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.287545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.309456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.310119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.310399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.310670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.311451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:46.311771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.313268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.327552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 527/1080] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:46.333408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.363908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.366056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.441732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d433-614d-43a9-be0e-b87b0147ccc1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.450430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.474787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 554/1081] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:46.480453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.482411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:46.482918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.484692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.484981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.485204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/92650894c3fb4749b87bd354b92e58de` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.487543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=a1fb1196840240b4aa05a2f83d4c7ee7, role_id=92650894c3fb4749b87bd354b92e58de)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.489490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.505101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 528/1082] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:46 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/a1fb1196840240b4aa05a2f83d4c7ee7/roles/92650894c3fb4749b87bd354b92e58de => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:20:46.599351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931156ed-50e3-4dfd-9b11-f5a3afa276c9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.640958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.641887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.642165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.642434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.654950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.656406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-11caaa07-7e39-4dc7-b2ac-1269ad0d7398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:46.676434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:46.678248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:46.679140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:46.679140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:46.679140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:46.679140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:46.679140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 555/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:46.684297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:46.788719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ff662-64c3-4296-bace-b1e7bf10488f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:46.834805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:46.835478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:46.835763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:46.836042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:46.836774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:46.837083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:46.838502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056e49a-1695-48b6-9b42-0c6f3467247e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:46.857383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 529/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:48.169230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128383-6ecf-4052-8ffa-7213b892bb35 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.169816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128383-6ecf-4052-8ffa-7213b892bb35 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.169816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128383-6ecf-4052-8ffa-7213b892bb35 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.171436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 556/1085] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:48.176543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.176932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.177433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.194245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:48.195422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:48.215107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.262443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 530/1086] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:48.285035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4128383-6ecf-4052-8ffa-7213b892bb35 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.340724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.385541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4128383-6ecf-4052-8ffa-7213b892bb35 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.419999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.420735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.421017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.421292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.432081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ce438202-9db7-4109-b3ac-2326d1bfcae7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:48.439681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:48.440687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:48.441584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 557/1087] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:48.446800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.538512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f482-984a-40c3-8f6d-51995aff17a7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.538795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f482-984a-40c3-8f6d-51995aff17a7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.539069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9f482-984a-40c3-8f6d-51995aff17a7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.540572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 558/1088] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:20:48.544122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4c70f-e743-4507-aa88-1cc85c59f836 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.545200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.545525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.545809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.562904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:48.564093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:48.579133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.579794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.580078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.580353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.581089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:48.581429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.582784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.590765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 531/1089] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:48.596345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.614551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.631483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 559/1090] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:20:48.654665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.696918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efaa7-a3e6-41e7-9cdf-b11dad31bfa2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.701918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.731440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.732562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.732843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.733121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] PATH_INFO: `/v3/users/f61c2ef3c9ea40e599da260eb78e43bf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.734346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] RBAC: Authorizing `identity:get_user(user_id=f61c2ef3c9ea40e599da260eb78e43bf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.735766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.737424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 532/1091] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/users/f61c2ef3c9ea40e599da260eb78e43bf => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:48.743840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.758984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eea4d3-d35f-475b-aaa8-095e0d522d1c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.791908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.792720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.793001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.793286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.804006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:20:48.812114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:48.813242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:48.814159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 560/1092] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:48.819349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:48.829682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.846375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd3835-11ae-4eb4-99df-4e745f33adca None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.879446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.880092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.880396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.881155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.919346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c888385-8aa5-40be-bd14-a8622e7acd19 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:48.922336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:48.949475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.950808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.953064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:48.953765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:48.954047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:48.954325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:48.955067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:48.955370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:48.956775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:48.964698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 561/1093] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:48.970312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.033965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 533/1094] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:20:49.055345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.067299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eea203-3e16-48fc-b22e-4315f23dc15a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.101279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.102170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.102389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.102667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.113169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.114645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:49.132097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:49.133007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:49.133935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:49.133935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:49.133935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:49.133935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:49.133935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 562/1095] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:49.139715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.157497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d693c2-db83-4573-82a3-9dfb5c406ccc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.158756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.193591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.194452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.194738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.195014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.196238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.197712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.199762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 534/1096] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:49.205354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.239351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7824f0-1054-4687-993d-1a047399f361 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.264576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.273771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.274428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.274707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.274979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.275691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:49.275992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.277355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.289252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 563/1097] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:49.294831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.309076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befa922-67cd-4c4a-9313-f50e62266135 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.343632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.344271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.344552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.344825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.345677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=963a41c11d45405f80b4668c2d931163, user.id=f61c2ef3c9ea40e599da260eb78e43bf) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:49.345988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.347447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.365359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 535/1098] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/role_assignments?user.id=f61c2ef3c9ea40e599da260eb78e43bf&scope.project.id=963a41c11d45405f80b4668c2d931163&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:49.372534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.396436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a1fa-ae23-4fd6-9a7b-82cde40c02d6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.436663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.437513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.437797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.438069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.446556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.448066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:20:49.464296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:49.465655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:20:49.466696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 564/1099] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:49.471459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.574760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56531-316e-468d-81a2-a0aa160531bd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.592049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.609412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.610078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.610363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.610640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.611473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:49.611783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.613212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.630279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 536/1100] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:49.636419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:49.734429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726e482-c00f-4a9c-b38a-8718d9945aec None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:49.767723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:49.768370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:49.768652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:49.768926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:49.777215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:49.838381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:49.839830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:49.936347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 565/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:49.942039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.039528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc0b7-13af-481b-b78c-1fac508aef11 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.073295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.074101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.074383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.074659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.084497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.085963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:20:50.103244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:50.104306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:50.105225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:20:50.105225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:20:50.105225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:20:50.105225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:50.105225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 537/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:20:50.110064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.149372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.206376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a73e55-58da-45d7-b94c-fad1070ff22f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.238885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.239534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.240596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.240596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.241293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:50.241622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.242971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.253785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 566/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:50.259894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.287846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.358094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbadb72-7a2b-432a-a90e-17b43cb0c544 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.390803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.391578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.391863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.392147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.399231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.400663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:20:50.417007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:20:50.418085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:20:50.418997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 538/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:20:50.424010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.522599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.526974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2508-94f1-4183-bdaf-734a288c761e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.559676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.560330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.560614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.560892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.561704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:50.562011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.563424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7f5b9-a3bf-4ba7-a90c-5e792d4a48e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.575549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 567/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:20:50.581493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:50.668686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:50.682304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6942-9afb-4e01-a71a-0b749d950d94 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:50.721398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:50.722045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:50.722332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:50.722609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:50.723433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] RBAC: Adding query filter params (role.id=92650894c3fb4749b87bd354b92e58de, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=a1fb1196840240b4aa05a2f83d4c7ee7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:50.723746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:50.725222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6ebe-f8d9-4919-9c1a-d962ada652a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:50.730973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 539/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:50 2022] GET /identity/v3/role_assignments?user.id=a1fb1196840240b4aa05a2f83d4c7ee7&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=92650894c3fb4749b87bd354b92e58de => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:51.409784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf96b6-7aa2-4612-ba6a-2e935b828f3e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.410303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf96b6-7aa2-4612-ba6a-2e935b828f3e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.410303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf96b6-7aa2-4612-ba6a-2e935b828f3e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.412351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 568/1107] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:51.417824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.418226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.418392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.436314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:51.437547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:51.505202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 540/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:51.530696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf96b6-7aa2-4612-ba6a-2e935b828f3e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.631086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf96b6-7aa2-4612-ba6a-2e935b828f3e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.664969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.665718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.666001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.666276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.667438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.668854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49adf4b-f337-4658-87e3-12bef215fa83 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.670486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 569/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:51.676291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:51.759755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:51.778002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60385250-d0e9-42aa-bcb3-2021c31afe69 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:51.810634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:51.811280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:51.811562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:51.811837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:51.812528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:51.813965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f147744-72df-461b-a71b-39fd0f289542 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:51.833301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 541/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:20:51 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:20:53.885120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451434-bbab-424b-a696-d149152af2c2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.885670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451434-bbab-424b-a696-d149152af2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.885773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451434-bbab-424b-a696-d149152af2c2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.887070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 570/1111] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:53.892216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:53.892638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:53.892770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:53.909538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:53.910702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:53.975818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 542/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:53.999041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf451434-bbab-424b-a696-d149152af2c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.025949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.098517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf451434-bbab-424b-a696-d149152af2c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.132631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705288-a01f-4c85-94db-de258231748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.133601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705288-a01f-4c85-94db-de258231748a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.133887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705288-a01f-4c85-94db-de258231748a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.134167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705288-a01f-4c85-94db-de258231748a None admin] PATH_INFO: `/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.135695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5705288-a01f-4c85-94db-de258231748a None admin] RBAC: Authorizing `identity:get_role(role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.137016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5705288-a01f-4c85-94db-de258231748a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.138406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 571/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:53 2022] GET /identity/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.144145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.208221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.241680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4e078-24fc-48fb-bc8c-b676dad64dad None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.274686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.275530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.275808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.276081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] PATH_INFO: `/v3/users/3bbef4a5f51d47f5bc166f78895e248f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.289230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] RBAC: Authorizing `identity:get_user(user_id=3bbef4a5f51d47f5bc166f78895e248f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.290679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.292280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 543/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/users/3bbef4a5f51d47f5bc166f78895e248f => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.297969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705288-a01f-4c85-94db-de258231748a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.388311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.395096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705288-a01f-4c85-94db-de258231748a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.428750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.429431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.429718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.429992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.494321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.495743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.583489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.591820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 572/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:20:54.597674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.703147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138b44-c091-409a-aa24-934dc9efba09 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.729930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:54.748532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.749456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.749840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.750221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.751428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.753044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177205a5-c112-478c-a5bd-b90b4a2fe1a8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.754846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 544/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:54.761406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:54.856423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940a015-03fc-4c57-8d33-c7a69f9291f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:54.888910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:54.889594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:54.889860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:54.890141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:54.890964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=963a41c11d45405f80b4668c2d931163, user.id=3bbef4a5f51d47f5bc166f78895e248f) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:20:54.891271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:54.892691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d3ec-e781-451f-9b3a-82c4c764b0cb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:54.898013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 573/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:20:54 2022] GET /identity/v3/role_assignments?user.id=3bbef4a5f51d47f5bc166f78895e248f&scope.project.id=963a41c11d45405f80b4668c2d931163&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:56.910909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5f0bb-ddb8-43e0-968d-6aeb19d8166f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:56.911481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5f0bb-ddb8-43e0-968d-6aeb19d8166f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:56.911481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5f0bb-ddb8-43e0-968d-6aeb19d8166f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:56.912862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 545/1118] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:56.918114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:56.918498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:56.918675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:56.941935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:56.943771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:56.994897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:57.009138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 574/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:57.031228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5f0bb-ddb8-43e0-968d-6aeb19d8166f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:57.129660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5f0bb-ddb8-43e0-968d-6aeb19d8166f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:57.162622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:57.163395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.163677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.163952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] PATH_INFO: `/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.165421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] RBAC: Authorizing `identity:get_role(role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:57.166765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:57.168086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 546/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:57 2022] GET /identity/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:57.173819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:57.268218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ffb13-50fa-418e-b7ab-4fd327bf2739 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:57.300680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:57.301683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.301965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.302522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] PATH_INFO: `/v3/users/3bbef4a5f51d47f5bc166f78895e248f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.303697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] RBAC: Authorizing `identity:get_user(user_id=3bbef4a5f51d47f5bc166f78895e248f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:57.305112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:57.306749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 575/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:57 2022] GET /identity/v3/users/3bbef4a5f51d47f5bc166f78895e248f => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:57.312445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:57.372402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:57.410379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62823c85-5284-47e4-8c1f-a62006506e9f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:57.443951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:57.444790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.445069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.445362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.446515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:57.448009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70c1a3-4789-4565-9c96-40b71181ccab None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:57.449704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 547/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:20:57 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:57.455590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:57.510253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:57.552128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee88e59-359c-4749-bceb-a8c6e4a0566e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:57.585342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:57.586239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:57.586520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:57.586794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163/users/3bbef4a5f51d47f5bc166f78895e248f/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:57.588955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=963a41c11d45405f80b4668c2d931163, user_id=3bbef4a5f51d47f5bc166f78895e248f, role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:57.590419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb3a83-31cf-4513-8fde-2464280b5c11 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:57.602909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 576/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:20:57 2022] PUT /identity/v3/projects/963a41c11d45405f80b4668c2d931163/users/3bbef4a5f51d47f5bc166f78895e248f/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:20:59.572785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329270f-4c9a-4b1d-b685-34dd45e56941 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.573364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329270f-4c9a-4b1d-b685-34dd45e56941 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.573364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329270f-4c9a-4b1d-b685-34dd45e56941 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.574765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 548/1124] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:20:59.579909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.580306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.580458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.596642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:20:59.597841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:20:59.661007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 577/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:20:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:20:59.685210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329270f-4c9a-4b1d-b685-34dd45e56941 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.784429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329270f-4c9a-4b1d-b685-34dd45e56941 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.812024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.817944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.818791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.819085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.819364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] PATH_INFO: `/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.820829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] RBAC: Authorizing `identity:get_role(role_id=6e2dbbce7b7c4f9eae9ded941aebd696)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.822223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.823569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 549/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/roles/6e2dbbce7b7c4f9eae9ded941aebd696 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:59.829361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:20:59.874855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:20:59.926922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ebfbd-6fc9-466f-b42e-d2f0cbee7c98 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:20:59.960122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:20:59.960926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:20:59.961207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:20:59.961509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] PATH_INFO: `/v3/users/3bbef4a5f51d47f5bc166f78895e248f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:20:59.962685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] RBAC: Authorizing `identity:get_user(user_id=3bbef4a5f51d47f5bc166f78895e248f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:20:59.964106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:20:59.965742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 578/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/users/3bbef4a5f51d47f5bc166f78895e248f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:20:59.971693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:00.067329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd59912-dd19-4ae2-ade6-9023b7be637a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:00.099015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:00.100161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:00.100438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:00.100714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:00.160935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:00.165548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:00.166954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:00.261578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 550/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:20:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:00.267269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:00.363276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe4954-022b-4f8e-ac6e-9d302152738b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:00.376507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:00.397320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:00.398187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:00.398468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:00.398744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] PATH_INFO: `/v3/projects/963a41c11d45405f80b4668c2d931163` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:00.399904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] RBAC: Authorizing `identity:get_project(project_id=963a41c11d45405f80b4668c2d931163)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:00.401324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd9901-5139-459f-a9b1-47a218b1e497 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:00.403057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 579/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:00 2022] GET /identity/v3/projects/963a41c11d45405f80b4668c2d931163 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:00.408987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:00.502520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:00.507188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fd1e7-534f-45de-9eea-b76df1af2e20 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:00.540998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:00.541614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:00.541901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:00.542180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:00.542998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] RBAC: Adding query filter params (role.id=6e2dbbce7b7c4f9eae9ded941aebd696, scope.project.id=963a41c11d45405f80b4668c2d931163, user.id=3bbef4a5f51d47f5bc166f78895e248f) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:00.543311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:00.544747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f76c64-e7e3-4abe-995e-fb40ecfcd08b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:00.550378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 551/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:00 2022] GET /identity/v3/role_assignments?user.id=3bbef4a5f51d47f5bc166f78895e248f&scope.project.id=963a41c11d45405f80b4668c2d931163&role.id=6e2dbbce7b7c4f9eae9ded941aebd696 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.545883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6062a7-91a9-4418-a561-bc11157129a1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.546423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6062a7-91a9-4418-a561-bc11157129a1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.546423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6062a7-91a9-4418-a561-bc11157129a1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.547787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 580/1131] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:02.553576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.554024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.554124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.570464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:02.571627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:02.636036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 552/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:02.658721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6062a7-91a9-4418-a561-bc11157129a1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.758699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6062a7-91a9-4418-a561-bc11157129a1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.791163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.791946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.792237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.792509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.793710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.795135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc7d1-d18b-40bc-9ac3-d05e2c1009a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.796736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 581/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:02.803100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:02.897768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:02.906382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29cb1d-0155-4a8e-a560-f9812ecabd77 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:02.939456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:02.940099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:02.940382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:02.940717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:02.941376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:02.943653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c027d6-39ce-4f5d-8097-04c4f5dbca8f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:02.968228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 553/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:02 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:05.014584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2945-ab1e-47f9-8255-5c08a39cfdda None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.015396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2945-ab1e-47f9-8255-5c08a39cfdda None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.015396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2945-ab1e-47f9-8255-5c08a39cfdda None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.016463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 582/1135] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:05.022218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.022670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.022772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.043023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:05.045088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:05.112478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 554/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:05.138223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2945-ab1e-47f9-8255-5c08a39cfdda None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.188784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.237163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2945-ab1e-47f9-8255-5c08a39cfdda None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.268893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.269634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.269913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.270181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.271892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.273303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b702d-49e2-40ed-92d2-5f0b2c080925 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.274917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 583/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:21:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:05.280538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:05.352594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:05.377741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fa721-e4fa-42e4-88f2-fd18a344efdd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:05.411556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:05.412214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:05.412501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:05.412782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:05.413518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:05.414950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744db40-5e9c-4100-95e9-dad20d21ec6d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:05.435287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 555/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:21:05 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:07.477086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7196-abbb-4ee9-9e96-007b78b4773f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.477641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7196-abbb-4ee9-9e96-007b78b4773f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.477641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7196-abbb-4ee9-9e96-007b78b4773f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.479071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 584/1139] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:07.484338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.484721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.484893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.501900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:07.503067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:07.569036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 556/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:07.593188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e7196-abbb-4ee9-9e96-007b78b4773f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.696193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e7196-abbb-4ee9-9e96-007b78b4773f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.712289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.729275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.730082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.730361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.730632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.740888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:07.748393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:07.749599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:07.750705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 585/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:07.755422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:07.812025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:07.853282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40b2e-8343-445e-b17f-a4d67cabd99b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:07.886930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:07.887589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:07.887875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:07.888156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:07.888899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:07.889208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:07.890605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:07.899054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 557/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:07.904759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.002854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788af9af-0028-49a2-91b6-d62b1379fb6f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.038746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.039615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.039899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.040428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] PATH_INFO: `/v3/users/07f382288c2a40c685b40465aaa4ded6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.053350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] RBAC: Authorizing `identity:get_user(user_id=07f382288c2a40c685b40465aaa4ded6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.054802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.056458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 586/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:07 2022] GET /identity/v3/users/07f382288c2a40c685b40465aaa4ded6 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:08.062348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.159800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed8dfb-e6e2-44df-83ef-3dacb0637829 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.181626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.193569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.194199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.194478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.194751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.259217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.260575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.355388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 558/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:08.361280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.395200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.458255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7e00-006f-4e97-a09a-4b055b753149 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.490767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.491625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.491908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.492185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] PATH_INFO: `/v3/projects/a38db3e7cae5484eba4b0b400b4501b6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.493370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] RBAC: Authorizing `identity:get_project(project_id=a38db3e7cae5484eba4b0b400b4501b6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.494791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854aa21-42f2-4ea0-9b6b-ed5674f0b838 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.496476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 587/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/projects/a38db3e7cae5484eba4b0b400b4501b6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:08.502498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:08.526501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:08.599307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b293293-e3bb-4991-810f-d2149bde24eb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:08.632051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:08.632686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:08.632967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:08.633241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:08.634166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=a38db3e7cae5484eba4b0b400b4501b6, user.id=07f382288c2a40c685b40465aaa4ded6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:08.634476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:08.635886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc60bf-432b-442b-8ee1-9d7324ee3c4a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:08.641067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 559/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:08 2022] GET /identity/v3/role_assignments?user.id=07f382288c2a40c685b40465aaa4ded6&scope.project.id=a38db3e7cae5484eba4b0b400b4501b6&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:10.625179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b65e-e0a6-4c3f-ac38-19fa1644c216 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.625838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b65e-e0a6-4c3f-ac38-19fa1644c216 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.625838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b65e-e0a6-4c3f-ac38-19fa1644c216 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.627123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 588/1147] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:10.632948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.633410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.633528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.650610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:10.651788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:10.715545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 560/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:10.737479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b65e-e0a6-4c3f-ac38-19fa1644c216 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.835099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b65e-e0a6-4c3f-ac38-19fa1644c216 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:10.837343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.867701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:10.868475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:10.868756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:10.869030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:10.878845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:10.886960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:10.888055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:10.888919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 589/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:10.894319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:10.912346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:10.993173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae89f33-b50a-49d0-81e3-754a15481dc9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.026597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.027621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.027916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.028198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.028937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:11.029246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.030706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.039140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 561/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.045028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.153305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251faab-f7d9-410d-bdef-0b2264f66c2b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.185726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.186575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.186852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.187123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] PATH_INFO: `/v3/users/07f382288c2a40c685b40465aaa4ded6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.188285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] RBAC: Authorizing `identity:get_user(user_id=07f382288c2a40c685b40465aaa4ded6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.189724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.191320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 590/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/users/07f382288c2a40c685b40465aaa4ded6 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.197572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.274856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.295756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d42efa-3f19-4a88-b858-731354dfcd05 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.328303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.329153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.329465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.329745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] PATH_INFO: `/v3/projects/a38db3e7cae5484eba4b0b400b4501b6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.330913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] RBAC: Authorizing `identity:get_project(project_id=a38db3e7cae5484eba4b0b400b4501b6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.332340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb276c-f6e7-4c09-b497-b3f1b071f683 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.334570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 562/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:11 2022] GET /identity/v3/projects/a38db3e7cae5484eba4b0b400b4501b6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:11.340526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:11.371221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:11.438102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c01461-9495-4c53-8acc-c7cb21686d75 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:11.469723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:11.470603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:11.470879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:11.471148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] PATH_INFO: `/v3/projects/a38db3e7cae5484eba4b0b400b4501b6/users/07f382288c2a40c685b40465aaa4ded6/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:11.473082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] RBAC: Authorizing `identity:create_grant(project_id=a38db3e7cae5484eba4b0b400b4501b6, user_id=07f382288c2a40c685b40465aaa4ded6, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:11.475040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e40e5c-b6be-4126-97cb-833a24692785 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:11.487281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 591/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:11 2022] PUT /identity/v3/projects/a38db3e7cae5484eba4b0b400b4501b6/users/07f382288c2a40c685b40465aaa4ded6/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:13.484017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a858-c61a-448f-8ac6-e2aedbebc443 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.484492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a858-c61a-448f-8ac6-e2aedbebc443 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.484581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a858-c61a-448f-8ac6-e2aedbebc443 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.485957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 563/1154] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:13.491646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.492030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.492197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.509311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:13.510506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:13.574613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 592/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:21:13.597821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462a858-c61a-448f-8ac6-e2aedbebc443 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.699340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462a858-c61a-448f-8ac6-e2aedbebc443 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.732061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.732837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.733121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.733424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.743294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:13.750899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:13.751991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:21:13.753003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 564/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:13.758570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.782051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:13.855763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f4604-638b-47a2-9701-d9461f10f20b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:13.888090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:13.888746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:13.889023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:13.889295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:13.890053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:13.890357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:13.891705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:13.899571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 593/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:13.905212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:13.915808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.004827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd123d-3bbd-4f4d-8428-fec4ce13360b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.037291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.038158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.038445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.038721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] PATH_INFO: `/v3/users/07f382288c2a40c685b40465aaa4ded6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.039892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] RBAC: Authorizing `identity:get_user(user_id=07f382288c2a40c685b40465aaa4ded6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.041304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.042924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 565/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:13 2022] GET /identity/v3/users/07f382288c2a40c685b40465aaa4ded6 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:14.048566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.132226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.147627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd65cf-374c-40f5-8fc5-b4aeb8f5ff9a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.179899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.180535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.180893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.181171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.245193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.246659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.324870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.340604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 594/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:14.346484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.420673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:14.445544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4569387-b4d0-4044-a23e-1507981e5e8e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.477857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.478694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.478975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.479248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] PATH_INFO: `/v3/projects/a38db3e7cae5484eba4b0b400b4501b6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.480416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] RBAC: Authorizing `identity:get_project(project_id=a38db3e7cae5484eba4b0b400b4501b6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.481874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c71f7-78eb-4060-97bb-464c81850826 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.483554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 566/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/projects/a38db3e7cae5484eba4b0b400b4501b6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:14.490114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:14.583964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d448d6-366b-4426-b670-b861bcf24ca0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:14.616598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:14.617239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:14.617545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:14.617822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:14.618688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=a38db3e7cae5484eba4b0b400b4501b6, user.id=07f382288c2a40c685b40465aaa4ded6) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:14.618952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:14.620359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262d7b2-968a-422c-87e2-cb0a47579ee6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:14.626115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 595/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:14 2022] GET /identity/v3/role_assignments?user.id=07f382288c2a40c685b40465aaa4ded6&scope.project.id=a38db3e7cae5484eba4b0b400b4501b6&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:16.697542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec644c-51fb-438b-ae80-4da99477a122 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.698040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec644c-51fb-438b-ae80-4da99477a122 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.698040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec644c-51fb-438b-ae80-4da99477a122 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.699372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 567/1162] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:16.704452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.704856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.705000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.723094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:16.724274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:16.766966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:16.790432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 596/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:16.812725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec644c-51fb-438b-ae80-4da99477a122 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:16.910919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec644c-51fb-438b-ae80-4da99477a122 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:16.943598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:16.944225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:16.944508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:16.944782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:16.955979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.010608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.012226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.108739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 568/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:17.116666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.213021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211ac60-221e-4ced-8236-ecee9eaf90bb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.246588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.247514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.247802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.248113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.255341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.256874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e28703df-cb10-4592-b990-4f3cf6ba4a30 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:17.271427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:17.272537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:17.273407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 597/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:17.278594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:17.320356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:17.376943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45611720-78d9-49c3-a8b9-3acd0af25cf0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:17.410514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:17.411197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:17.411518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:17.411795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:17.412511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:17.412821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:17.414244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7b271-ffdc-41e2-b0d4-57766f0cb958 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:17.421388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 569/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Jul 12 14:21:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:19.448809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4e9a-5715-47a2-8d5e-b93d966f74a8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.449294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4e9a-5715-47a2-8d5e-b93d966f74a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.449409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4e9a-5715-47a2-8d5e-b93d966f74a8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.450761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 598/1167] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:19.455924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78782746-9d5a-4575-acff-10cff609d6ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.456300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78782746-9d5a-4575-acff-10cff609d6ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.456491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78782746-9d5a-4575-acff-10cff609d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.473639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-78782746-9d5a-4575-acff-10cff609d6ab None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:19.474856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-78782746-9d5a-4575-acff-10cff609d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:19.538940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 570/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:19.561552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4e9a-5715-47a2-8d5e-b93d966f74a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:19.601585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:19.660707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4e9a-5715-47a2-8d5e-b93d966f74a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:19.693035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:19.693646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:19.693925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:19.694197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:19.694913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:19.696263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082e3f0-f90c-446e-ba7d-a7ce05060199 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:19.710731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 599/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Tue Jul 12 14:21:19 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:21.745133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad05ab-30f8-48b3-92bd-b1e9e9944562 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.745666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad05ab-30f8-48b3-92bd-b1e9e9944562 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.745666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad05ab-30f8-48b3-92bd-b1e9e9944562 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.747001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 571/1170] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:21.752059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.752452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.752619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:21.769018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:21.770239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:21.835170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 600/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:21.858867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad05ab-30f8-48b3-92bd-b1e9e9944562 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:21.893813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:21.957906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad05ab-30f8-48b3-92bd-b1e9e9944562 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:21.990243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:21.991080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:21.991361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:21.991635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] PATH_INFO: `/v3/domains/93ee2634e02b4daab2a2ff37e652a650` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.001680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=93ee2634e02b4daab2a2ff37e652a650)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.003044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685562db-3401-4752-aaa7-b30c4f386760 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.004631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 572/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Jul 12 14:21:21 2022] GET /identity/v3/domains/93ee2634e02b4daab2a2ff37e652a650 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:22.010860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:22.037586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:22.106677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6af250-ed7c-4975-8a4a-eaf509836c23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:22.140078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:22.140719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:22.140997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:22.141268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:22.141988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:22.143400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9faaad-8239-4a89-b35a-dd7d8b242cf9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:22.168258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 601/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:21:22 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:24.219513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becade9f-599e-4455-9cb9-9a952e6197cd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.220049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becade9f-599e-4455-9cb9-9a952e6197cd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.220049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becade9f-599e-4455-9cb9-9a952e6197cd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.221468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 573/1174] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:24.227145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.227516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.227714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.244740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:24.245950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:24.310042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 602/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:24.334106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becade9f-599e-4455-9cb9-9a952e6197cd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.428370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.435122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becade9f-599e-4455-9cb9-9a952e6197cd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.467560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.468405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.468688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.468965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] PATH_INFO: `/v3/domains/93ee2634e02b4daab2a2ff37e652a650` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.470212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=93ee2634e02b4daab2a2ff37e652a650)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.471579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b380e-08de-4250-8def-8ebbbb63a7fd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.473172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 574/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Jul 12 14:21:24 2022] GET /identity/v3/domains/93ee2634e02b4daab2a2ff37e652a650 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:24.479489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:24.490665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:24.576005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea3970-9fd4-4ea8-b238-e1f7edcd0def None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:24.607609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:24.608255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:24.608534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:24.608808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:24.609548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:24.610963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de277a08-d48b-4efb-9121-bc79d13e6085 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:24.631317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 603/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:21:24 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:21:26.672358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6188-4acd-4f30-b85e-52dc9e3c0eaa None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.673006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6188-4acd-4f30-b85e-52dc9e3c0eaa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.673006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f6188-4acd-4f30-b85e-52dc9e3c0eaa None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.674115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 575/1178] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:26.679647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.680046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.680202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.697007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:26.698226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:26.755717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:26.763769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 604/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:26.786352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6188-4acd-4f30-b85e-52dc9e3c0eaa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:26.884071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f6188-4acd-4f30-b85e-52dc9e3c0eaa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:26.915709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:26.916470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:26.916751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:26.917026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:26.928053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:26.935566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:26.936819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:21:26.937942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 576/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:21:26.943251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.038319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7190463-a9d3-4d48-b42b-88f461e132af None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.070723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.071376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.071659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.071933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.072666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.072972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.074365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.082207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 605/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.088520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.114557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.194035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40262d5e-5895-4ae5-beff-7e549f263e68 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.226996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.227844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.228129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.228405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] PATH_INFO: `/v3/users/e672b3b7886d4cc39658f179ae0c3aac` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.241095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] RBAC: Authorizing `identity:get_user(user_id=e672b3b7886d4cc39658f179ae0c3aac)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.242556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.244183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 577/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/users/e672b3b7886d4cc39658f179ae0c3aac => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.250076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.314990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.355259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e20c-3f84-4278-8afd-1de2e0ae847a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.387223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.387860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.388141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.388416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.453107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.454525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.508722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.550182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 606/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:27.556929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.635781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:27.653382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e8268-add3-4442-8e79-d04c813698ec None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.686285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.687121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.687405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.687682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] PATH_INFO: `/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.688839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] RBAC: Authorizing `identity:get_project(project_id=d0f0c6e6a4c9409c8934c166cf11385f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.690277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1aec28-546f-4f75-83e7-f93cfdcd388f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.691937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 578/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:27.698526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:27.791678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6934202-aff8-4e00-a987-1a0029ce9ea8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:27.824825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:27.825489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:27.825767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:27.826039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:27.826854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=d0f0c6e6a4c9409c8934c166cf11385f, user.id=e672b3b7886d4cc39658f179ae0c3aac) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:27.827160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:27.828575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d88e1-c101-4c79-8b80-5adac550ac65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:27.833843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 607/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:27 2022] GET /identity/v3/role_assignments?user.id=e672b3b7886d4cc39658f179ae0c3aac&scope.project.id=d0f0c6e6a4c9409c8934c166cf11385f&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:29.831039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7b93-50fb-4ceb-80d3-6a8c98045730 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.831535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7b93-50fb-4ceb-80d3-6a8c98045730 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.831535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7b93-50fb-4ceb-80d3-6a8c98045730 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.832862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 579/1186] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:29.837968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:29.838362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:29.838526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:29.854867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:29.856048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:29.872799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:29.921537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 608/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:29.944758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f7b93-50fb-4ceb-80d3-6a8c98045730 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.042422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f7b93-50fb-4ceb-80d3-6a8c98045730 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.075446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.076209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.076498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.076779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.087210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:30.094770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:30.095890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:21:30.096911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 580/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:30.102749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.199700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b2719-7d30-4d2c-bf99-b11d81edd4a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.232684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.233365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.233660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.233939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.234678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:30.234986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.236345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.244794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 609/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.250454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.262963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.347870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db689401-9f7c-4321-ab68-7f17f2e4e971 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.380646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.381525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.381811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.382086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] PATH_INFO: `/v3/users/e672b3b7886d4cc39658f179ae0c3aac` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.383264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] RBAC: Authorizing `identity:get_user(user_id=e672b3b7886d4cc39658f179ae0c3aac)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.384677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.386303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 581/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/users/e672b3b7886d4cc39658f179ae0c3aac => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.392657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.409232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.491472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157b32-cb93-4c9d-9bd1-33c4f5554536 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.523414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.524264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.524546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.524820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] PATH_INFO: `/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.525998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] RBAC: Authorizing `identity:get_project(project_id=d0f0c6e6a4c9409c8934c166cf11385f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.527418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351479df-2703-49d5-bb88-5e7f95ab8e48 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.529103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 610/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:30 2022] GET /identity/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:30.534985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:30.611641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:30.632827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf571d8b-2f55-4c8e-b393-1a76548d8365 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:30.665481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:30.666408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:30.666692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:30.666966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] PATH_INFO: `/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f/users/e672b3b7886d4cc39658f179ae0c3aac/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:30.668882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0f0c6e6a4c9409c8934c166cf11385f, user_id=e672b3b7886d4cc39658f179ae0c3aac, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:30.670316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4d5f9-9239-4bb2-bc54-6151bd11f975 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:30.682765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 582/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:21:30 2022] PUT /identity/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f/users/e672b3b7886d4cc39658f179ae0c3aac/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:21:32.758668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc615-eed8-4e2b-b10c-e0f06ad66893 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.759219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc615-eed8-4e2b-b10c-e0f06ad66893 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.759219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc615-eed8-4e2b-b10c-e0f06ad66893 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.760499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 611/1193] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:21:32.765733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:32.766156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:32.766301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:32.783272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:21:32.784469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:21:32.849998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 583/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:21:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:21:32.873631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdc615-eed8-4e2b-b10c-e0f06ad66893 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:32.953668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:32.972857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdc615-eed8-4e2b-b10c-e0f06ad66893 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.006126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.006910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.007192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.007465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.017380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:21:33.025551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:21:33.026637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:21:33.027675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 612/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:21:33.032764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.095801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.129147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87576c43-2eea-4998-8fbd-63d83937612e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.163397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.164051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.164331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.164650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.165451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.165760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.167118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.175681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 584/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:21:33.181796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.287488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8293c39-26df-40a9-ace0-03d92a8072af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.322363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.323214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.323486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.323752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] PATH_INFO: `/v3/users/e672b3b7886d4cc39658f179ae0c3aac` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.324918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] RBAC: Authorizing `identity:get_user(user_id=e672b3b7886d4cc39658f179ae0c3aac)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.326372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.327956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 613/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/users/e672b3b7886d4cc39658f179ae0c3aac => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.334047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.435557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105b69-34f6-4ec7-87da-eb06318b08bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.464139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.470369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.471011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.471298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.471575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.537826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.539192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.634819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 585/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:21:33.640771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.653792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.743344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db08eb-f9c6-4a93-ae05-f42ca66d48af None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.776010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.776865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.777151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.777458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] PATH_INFO: `/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.778642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] RBAC: Authorizing `identity:get_project(project_id=d0f0c6e6a4c9409c8934c166cf11385f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.780069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc27d30-3c2a-4c8c-88b4-5d054b074c2d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.781793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 614/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/projects/d0f0c6e6a4c9409c8934c166cf11385f => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:33.788447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:21:33.817599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:33.885109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75bfe2-af8d-4390-b4bb-68c5dd4a8613 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:21:33.917766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:21:33.918405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:21:33.918687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:21:33.918960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:21:33.919781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=d0f0c6e6a4c9409c8934c166cf11385f, user.id=e672b3b7886d4cc39658f179ae0c3aac) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:21:33.920089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:21:33.921517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57337fbd-49c7-4af3-8e2d-b262ef90ca8d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:21:33.927148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 586/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:21:33 2022] GET /identity/v3/role_assignments?user.id=e672b3b7886d4cc39658f179ae0c3aac&scope.project.id=d0f0c6e6a4c9409c8934c166cf11385f&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:21:43.771870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=61 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:21:43.925309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=53 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:29.149007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8120c-3458-4e91-a635-ae986ed14357 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:29.149643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8120c-3458-4e91-a635-ae986ed14357 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:29.149643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8120c-3458-4e91-a635-ae986ed14357 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:29.151213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 615/1201] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:29.158008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:29.158652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:29.158907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:29.179650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:29.180864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:29.267812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 587/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:29.297062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8120c-3458-4e91-a635-ae986ed14357 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:29.432035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8120c-3458-4e91-a635-ae986ed14357 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:29.474260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:29.475341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:29.475833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:29.476411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:29.490152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:29.492174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:29.514198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:29.515475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:29.516434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:29.516434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:29.516434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:29.516434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:29.516434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 616/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:29.522518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:29.642947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120203a-5e0f-4468-95e8-2ff14ce688d5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:29.675868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:29.684596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:29.686407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:29.686407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:29.686407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:29.687296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:29.687650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:29.689520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd07d5f-0d74-45b7-b2fa-87baba27e949 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:29.705384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 588/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:29.709923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:29.783858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:29.838228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58468-7cea-4141-b92a-fcda9f685412 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:29.877758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:29.878588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:29.878915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:29.879219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:29.879923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:29.881522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ee374-41b5-4886-9ff4-9c20df7ecb4a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:29.904906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 617/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Jul 12 14:22:29 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:22:32.208151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60293934-6dbb-4f9b-9dcb-e3c166fb39e1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:32.208694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60293934-6dbb-4f9b-9dcb-e3c166fb39e1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:32.208694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60293934-6dbb-4f9b-9dcb-e3c166fb39e1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:32.210547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 589/1206] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:32.218762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:32.218762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:32.218762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:32.235633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:32.237137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:32.324065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 618/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:32.350526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60293934-6dbb-4f9b-9dcb-e3c166fb39e1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:32.475003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60293934-6dbb-4f9b-9dcb-e3c166fb39e1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:32.517047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:32.517974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:32.519263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:32.519263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:32.531342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:32.539676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:32.540650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:32.542006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 590/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:22:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:32.547498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:32.593912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:32.669798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997174f-43fd-4c19-90d1-36c153e73fb5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:32.712260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:32.713007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:32.713315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:32.713617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:32.714436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:32.714853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:32.716497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:32.728674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 619/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:22:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:32.734387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:32.795218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:32.852797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59032ac-416f-40c3-85e5-9f1f1974cfb4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:32.891892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:32.892714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:32.892974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:32.893255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:32.904931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:32.906490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:32.927505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:32.929084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:32.930535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:32.930535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:32.930535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:32.930535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:32.930535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 591/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:32.935311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.056861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:33.060759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d234-d972-46a3-852c-1e81447ed862 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.101771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.102578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.102860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.103139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.103882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:33.104192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:33.105624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:33.121324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 620/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:33.126966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.256774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32117d-4245-4122-be65-31e761c0f50c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.258563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:33.299095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.299898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.300233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.300511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.308382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:33.310568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:33.325549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:33.326555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:33.327612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 592/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:33.332983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.441456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c911c-80e0-4e4c-b2ea-df485155067a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.479493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.480215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.480515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.480840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.481856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:33.482154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:33.484162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:33.504486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 621/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:33.510917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.633614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac34d83-34fc-4165-a096-2960a078a20d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:33.678171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:33.679403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:33.679403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:33.679610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:33.714623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:33.762120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:33.763831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:33.895824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 593/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:33.901609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:33.924538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.024033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057b717-c14b-47c9-b60f-4d0ff5aa4ba8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.065267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.066805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.067208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.067651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.080937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.082769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:34.103497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:34.104745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:34.106174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:34.106174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:34.106174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:34.106174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:34.109956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 622/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:34.111389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.187951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.241146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c8f7-2340-4608-bd4f-e207c9888891 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.283745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.284544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.284870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.285156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.285927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:34.286936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.287767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.304143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 594/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:34.309989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.399630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.436456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26affd25-ec25-4726-86e5-e6cea434dd99 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.474039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.474757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.475037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.475307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.482584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.484054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:34.501241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:34.502254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:34.508445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 623/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:34.508945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.617040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b4961-28e5-4986-a129-40a323cb9417 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.643148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.657900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.658560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.658844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.659284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.660189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:34.660503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.661980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9157b-2cee-49e8-a84d-b113b2b01d8b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.681195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 595/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:34.682412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:34.797387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884dd65-5968-4d2b-8571-6f32d3ed973f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:34.826964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:34.836118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:34.836780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:34.837057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:34.837354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:34.838185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=4a342fa16a564ef38675936d7f51578b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:34.838497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:34.839924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2bf4-a929-45e9-a3ca-619ca1bacb88 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:34.846522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 624/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:34 2022] GET /identity/v3/role_assignments?user.id=4a342fa16a564ef38675936d7f51578b&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:36.957548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc2fe-dfb7-4a2d-89d4-96ce6a865be5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.958640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc2fe-dfb7-4a2d-89d4-96ce6a865be5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.958640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc2fe-dfb7-4a2d-89d4-96ce6a865be5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.960923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 596/1220] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:36.965795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:36.966238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:36.966329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:36.983871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:36.985113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:37.059875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 625/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:37.084973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc2fe-dfb7-4a2d-89d4-96ce6a865be5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.205154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc2fe-dfb7-4a2d-89d4-96ce6a865be5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.245001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.245880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.247479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.247479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.258509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:37.269665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:37.271014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:37.272062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 597/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:37.279582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.400134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e69-22e3-4aff-9903-e1b7a07b8f64 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.431024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.443786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.444472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.444797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.445095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.445992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:37.446339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.447839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.456428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 626/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:37.462260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.508903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.571738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b96dba-4437-4b5b-a397-1fcc02a872b2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.607777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.608604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.608888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.609164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.622527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.624431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:37.645810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:37.647091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:37.648208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:37.648208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:37.648208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:37.648208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:37.648208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 598/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:37.653356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.767629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991a77d-e2dd-4562-a6fb-06e22147d50e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.804130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.804806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.805085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.805378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.806101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:37.806407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.807768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.819998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 627/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:37.826071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:37.924816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:37.932827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21967c09-d90a-4409-992f-a6fd9d75c72a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:37.970192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:37.971001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:37.971279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:37.971553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:37.979627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:37.981057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:37.996002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:37.997360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:37.998306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 599/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:38.003971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.056716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:38.111230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a74ff-b246-4b9e-b2ba-37358b0ccd80 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:38.147400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:38.148051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:38.148425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:38.148649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:38.149628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:38.149999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:38.151359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:38.172524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 628/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:38.178458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.284082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59fc79-4115-4e14-b36f-9e104dc09ea3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:38.321914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:38.322721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:38.323066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:38.323376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:38.334790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:38.336204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:38.355319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:38.356475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:38.362186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:38.362186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:38.362186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:38.362186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:38.362186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 600/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:38.362740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.476399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141670c-c9ec-4cc7-a1b3-1ff62b9a2117 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:38.480058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:38.516114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:38.516776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:38.517056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:38.517326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:38.518130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:38.518431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:38.520404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:38.532426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 629/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:38.538534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.549481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:38.658860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401831-43ef-4186-aa5b-7601573fbf03 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:38.698488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:38.699496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:38.699960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:38.700414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:38.709709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:38.711156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:38.727542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:38.728682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:38.729812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 601/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:38.736615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.848464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f738c-9cf6-4924-a33b-d56f492010dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:38.884408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:38.885109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:38.885410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:38.885697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:38.886524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:38.886869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:38.888314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec5fc-2cdd-4df0-b5c4-049d00596d09 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:38.901084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 630/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:38.907407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:38.987351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:39.024683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dfc8c-31f3-44a6-8f52-cec3feeb7074 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:39.065836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:39.067065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:39.067478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:39.067864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/4a342fa16a564ef38675936d7f51578b/roles/9c9e180a5656470392d87b1027126c65` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:39.085887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=4a342fa16a564ef38675936d7f51578b, role_id=9c9e180a5656470392d87b1027126c65)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:39.087462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f6d6d-aeae-49e0-98ea-a762e4171fb8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:39.100725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 602/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:38 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/4a342fa16a564ef38675936d7f51578b/roles/9c9e180a5656470392d87b1027126c65 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:22:41.252306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1036-4d66-40ac-8c93-941ed268f3a2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:41.252861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1036-4d66-40ac-8c93-941ed268f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:41.253037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1036-4d66-40ac-8c93-941ed268f3a2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:41.255380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 631/1233] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:41.260952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:41.261531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:41.261640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:41.280283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:41.281553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:41.353556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 603/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:41.377228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb1036-4d66-40ac-8c93-941ed268f3a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:41.405479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:41.489640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb1036-4d66-40ac-8c93-941ed268f3a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:41.525429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:41.526148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:41.526431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:41.526700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:41.538105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:41.545965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:41.547110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 14:22:41.548165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:41.549812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 632/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Jul 12 14:22:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:41.553364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:41.601574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:41.658497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f43b2-849d-45fa-b85e-266c8b917960 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:41.694379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:41.695105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:41.695386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:41.695658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:41.696392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:41.696695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:41.698376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:41.708391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 604/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Jul 12 14:22:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:41.714187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:41.801298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:41.819718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fadf2-adc1-48ed-9e69-bd1a592a021e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:41.854290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:41.855135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:41.855413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:41.855692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:41.867268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:41.868722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:41.887860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:41.888847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:41.890061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:41.890061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:41.890061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:41.890061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:41.890061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 633/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:41.895679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.003477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf4884-80c3-4d42-aea7-fc7c145213df None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.013902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:42.044466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.045295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.045622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.045906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.046760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:42.047209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.049029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.063759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 605/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:42.071664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.194950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f6ea-0b4e-484c-9e0b-722ca48cf36f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.232661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.233484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.233775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.234071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.237429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:42.243954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.245642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:42.261910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:42.263511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:42.264672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 634/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:42.269015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.385717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88842178-5635-42e0-83bb-8a5cc2c9c6d6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.421997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.422674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.422961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.423258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.424106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:42.424417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.426494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.445862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 606/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:42.451741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.558645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a5ad-ab36-4d65-8846-67d625dbd1c9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.594199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.594940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.595223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.595498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.670011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:42.676981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.678474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.790514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 635/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:42.796380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:42.827698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:42.909796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271948c-5b68-4184-90de-632d177962c2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:42.948939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:42.949807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:42.950111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:42.950466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:42.962219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:42.963806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:42.982611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:42.983696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:42.984674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:42.984674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:42.984674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:42.984674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:42.984674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 607/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 14:22:42.990144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:43.094689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:43.101236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54582ff5-b99f-494b-8475-f1f844db7e19 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:43.140185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:43.140903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:43.141195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:43.141565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:43.142387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:43.142706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:43.144837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:43.163032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 636/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:43.164345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:43.259734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:43.286607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50387-af21-4f3b-8936-5e80633f20d0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:43.322777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:43.323596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:43.323884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:43.324164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:43.332418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:43.334152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:43.357006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:43.358150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:43.364040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 608/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:43.366328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:43.479482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713a5e-17ca-44e8-9cb5-cb31c90035c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:43.516841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:43.517467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:43.517754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:43.518030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:43.518821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:43.519131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:43.520566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc76ed0-2403-4140-b3f7-26e75f6837b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:43.534308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 637/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:43.540385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:43.655042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff059c7f-1af4-4e51-9847-171e896ffde4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:43.692345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:43.693045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:43.693358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:43.693637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:43.694469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] RBAC: Adding query filter params (role.id=9c9e180a5656470392d87b1027126c65, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=4a342fa16a564ef38675936d7f51578b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:43.694783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:43.696283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ace0e0-b68c-4a89-b11d-e285b4dd957b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:43.702342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 609/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:43 2022] GET /identity/v3/role_assignments?user.id=4a342fa16a564ef38675936d7f51578b&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=9c9e180a5656470392d87b1027126c65 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:45.976205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489043-5ba9-4eac-a615-bfbcb48db7fe None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:45.976779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489043-5ba9-4eac-a615-bfbcb48db7fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:45.976910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49489043-5ba9-4eac-a615-bfbcb48db7fe None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:45.978305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 638/1247] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:22:45.984025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:45.984461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:45.984665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:45.990175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:46.003887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:46.005089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:46.077929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 610/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:46.107238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489043-5ba9-4eac-a615-bfbcb48db7fe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:46.127301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:46.224266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49489043-5ba9-4eac-a615-bfbcb48db7fe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:46.263117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:46.263848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:46.264130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:46.264472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:46.275560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:46.283580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:46.289893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:46.290918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 639/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:46.291755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:46.396719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cbfba-aaf0-4811-9752-a19bfcb4f7c8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:46.427987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:46.434207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:46.434910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:46.435223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:46.435545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:46.436469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:46.436778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:46.438293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:46.448394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 611/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:46.454510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:46.547782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:46.574239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c7ec-dcdf-49b5-b385-3516a699b171 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:46.612043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:46.613186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:46.613540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:46.613841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:46.626623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:46.628615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:46.648862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:46.650022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:46.650928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:46.650928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:46.650928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:46.650928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:46.650928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 640/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:46.655937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:46.767950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c9b2f-9b8d-45db-9ce4-1989e5e304a7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:46.805663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:46.806469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:46.806888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:46.807193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:46.808168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:46.808505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:46.810115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:46.825780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 612/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:46.833259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:46.941051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3d73a-16f2-4636-84ca-d44fdce38ba0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:46.970609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:46.980175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:46.980986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:46.981266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:46.981576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:46.990428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:46.992001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:47.006680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:47.008217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:47.009374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 641/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:47.013873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.079650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.128972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2e51b-15c4-49af-935b-67d368e34a7b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.168347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.169014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.169305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.169617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.170447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:47.170753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.172216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.196893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 613/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:47.198148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.306752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c0d0-2efb-49af-aafb-db94dead930f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.345003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.345771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.346056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.346338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.407226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.423981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.426071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.536203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 642/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 14:22:47.542913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.655009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecd886-51bc-43a3-9be6-6e49e05e9ff4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.667989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.691651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.692444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.692730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.693003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.704783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.707432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:47.729710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:47.730906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:47.731955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:47.731955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:47.731955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:47.731955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:47.733387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 614/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:47.737844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:47.849640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:47.864419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316629ba-8c0a-4352-b5e9-d268fe178d4f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:47.902816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:47.903405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:47.903706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:47.903954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:47.904656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:47.904962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:47.906434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:47.920731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 643/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:47.926883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:48.035774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e0d0d-7ad1-43d2-a2d6-03026d8cee5a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:48.076639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:48.077492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:48.077845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:48.078096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:48.085643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:48.087082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:48.104082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:48.105184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:48.106314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:48.110507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 615/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:48.111457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:48.235914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1508e3-3dab-4f59-87e3-fa95cda91c9b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:48.274381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:48.275237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:48.275575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:48.275921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:48.276948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:48.277289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:48.278940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656b676-3dc9-4bcb-81c0-e4b6996d7252 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:48.292789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 644/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:48.299413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:48.327323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:48.409312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0db94-6157-405e-95d9-c4f82c7ca8ba None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:48.446495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:48.447172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:48.447454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:48.447726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:48.448541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=4a342fa16a564ef38675936d7f51578b) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:48.448848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:48.450354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2d5d8-eb30-43ae-83d9-747168125bf4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:48.455740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 616/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:48 2022] GET /identity/v3/role_assignments?user.id=4a342fa16a564ef38675936d7f51578b&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:50.473873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55176-d6b5-43af-b7f0-1f8d71cccd80 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.474382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55176-d6b5-43af-b7f0-1f8d71cccd80 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.474472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55176-d6b5-43af-b7f0-1f8d71cccd80 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.477598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 645/1261] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:50.481147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.481581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.481743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.498643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:50.499809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:50.565119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 617/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 14:22:50.588219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb55176-d6b5-43af-b7f0-1f8d71cccd80 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.599159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.686844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb55176-d6b5-43af-b7f0-1f8d71cccd80 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.719659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.720439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.720724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.721000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.731174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:50.739191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:50.740288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:50.741639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 646/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:50.746457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.764462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:50.844676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53245f-da00-4fa0-b6c5-72676c7cbd61 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:50.880582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:50.881227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:50.881534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:50.881857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:50.882536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:50.882839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:50.884177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:50.891905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 618/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:50.897795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:50.963641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:51.000769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486512d-be0a-4f54-8e01-17d02dbe01f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.035745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.036527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.036809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.037087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.047686 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.049136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:51.066027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:51.067036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:51.067981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:51.067981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:51.067981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:51.067981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:51.067981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 647/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:51.073222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.161052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:51.185648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8928-1c32-4e38-b6b4-2fda07fdfdd9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.228235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.228820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.229107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.229417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.230168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:51.230478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.231840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.243376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 619/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:51.252370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.352395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dad04-a0b2-462c-9392-859dc9d95d23 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.371686 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:51.385437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.386236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.386517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.386786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.394134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.395568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:51.410036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:51.411176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:51.412485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 648/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:51.417242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.514268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a7c0e-b909-49b3-9fc7-37a18bb56e8f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.548126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.548783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.549067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.549353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.550724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:51.551037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.552459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.561230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:51.570384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 620/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:51.575972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.672884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e53d9d-592b-4bda-b886-45b1213e8b29 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.706330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.707123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.707404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.707675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.717844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.719278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:51.735666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:51.736685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:51.738329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:51.738329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:51.738329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:51.738329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:51.738329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 649/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:51.743046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.840015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc27673-aac8-4715-b925-94487b93e2f7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:51.873201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:51.873878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:51.874165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:51.874444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:51.875159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:51.875469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:51.876824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:51.888337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 621/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:51.894486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:51.925872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:51.993078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c3f9d-3752-41d7-a9d4-e0eb38ee32ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:52.026257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:52.027040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:52.027323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:52.027599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:52.034135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:52.035560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:52.049386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:52.050406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:52.051497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 650/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:52.056627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:52.115167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:52.162659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f550bdc-f029-40b7-a9d9-a64bf3d023c8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:52.199806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:52.200461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:52.200745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:52.201020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:52.201828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:52.202139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:52.203563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb4bcd-6166-4761-b028-e70910ce886a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:52.216537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 622/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:52.223220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:52.320682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:52.329671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083b86-71d9-401f-bc1b-a5a1572e5000 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:52.362948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:52.363832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:52.364114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:52.364391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] PATH_INFO: `/v3/projects/35a95557d6a5450399d77fd04fed233c/users/4a342fa16a564ef38675936d7f51578b/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:52.366422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a95557d6a5450399d77fd04fed233c, user_id=4a342fa16a564ef38675936d7f51578b, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:52.367788 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f137396-43c3-43e6-ad33-6d7e11c18df0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:52.380266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 651/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Jul 12 14:22:52 2022] PUT /identity/v3/projects/35a95557d6a5450399d77fd04fed233c/users/4a342fa16a564ef38675936d7f51578b/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:22:54.372337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732aab-5579-4828-88d7-b971a752305a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:54.372870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732aab-5579-4828-88d7-b971a752305a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:54.372870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79732aab-5579-4828-88d7-b971a752305a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:54.374267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 623/1274] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:54.379951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:54.380375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:54.380510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:54.397710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:54.398901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:54.464724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 652/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:54.488457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79732aab-5579-4828-88d7-b971a752305a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:54.587373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79732aab-5579-4828-88d7-b971a752305a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:54.603686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:54.620496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:54.621276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:54.621598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:54.621877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:54.632304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 14:22:54.639766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:54.641294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:54.642337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 624/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Jul 12 14:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:54.647647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:54.725613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:54.744147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35b693-70ac-4d7a-9caf-4b7c3a6fa0cf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:54.776564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:54.777216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:54.777523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:54.777799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:54.778535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:54.778837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:54.780191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:54.788164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 653/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Jul 12 14:22:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:54.793785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:54.895923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933947e-07f2-4ce0-a9f7-59f1b8d07b49 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:54.927976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:54.928751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:54.929037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:54.929595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:54.939977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:54.941975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:54.959614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:54.960478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:54.961400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:54.961400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:54.961400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:54.961400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:54.961400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 625/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:54.966513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:55.062483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96c91f-c8db-47ea-b7d2-4494df3138e5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:55.095190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:55.095838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.096115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.096383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.097639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:55.097944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:55.099294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:55.110565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 654/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:55.118109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:55.152240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:55.224709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97195d6-5ad2-42c5-8625-600912338da0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:55.257624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:55.258414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.258697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.258972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.266510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:55.267941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 14:22:55.282460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:55.283593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:55.285367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 626/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:55.290009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:55.293805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:55.386015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af403b-3954-46da-b5d3-3078efabcc68 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:55.419879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:55.420551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.420833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.421109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.421979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:55.422292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:55.423700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:55.440658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 655/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:55.446949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:55.543030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:55.553022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f3431-deed-49f0-ab98-cabdc6ebbe02 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:55.588470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:55.589095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.589401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.589678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.657760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:55.659227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:55.748136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:55.759144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 627/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 14:22:55.764982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:55.824565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:55.863947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87484025-1792-435e-a086-a3e5f656b143 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:55.898063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:55.899272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:55.899554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:55.899822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:55.911712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:55.913134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 14:22:55.931256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:55.932299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:55.933209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 14:22:55.933209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 14:22:55.933209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 14:22:55.933209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:55.933209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 656/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:55.939642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.039445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a1dc-f879-4a82-8ab9-681343bc926d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.073712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.074755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.075229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.075689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.076704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:56.077220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.079073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.090125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 628/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Jul 12 14:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:56.095804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.200871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c376ce-28ef-4fb0-b2c6-06ea60ca7fff None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.214887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.234734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.235513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.235789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.236056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.242680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.244096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 14:22:56.259137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:56.260204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Jul 12 14:22:56.261240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 657/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:56.266109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.315266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:56.364395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7480-33d8-43ed-a05d-aeaa11dde4d4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.401519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.402181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.402460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.402734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.403510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:56.403816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.405230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0db589-2c83-48a6-b2ae-6af630f8d24e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.417572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 629/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:22:56.423444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:56.525706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7cdeb-ad68-43c8-942f-ca6aee1da72e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:56.558079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:56.558723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:56.559004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:56.559276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:56.560097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] RBAC: Adding query filter params (role.id=640b98e8829b4de0bdceee05e74bf473, scope.project.id=35a95557d6a5450399d77fd04fed233c, user.id=4a342fa16a564ef38675936d7f51578b) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:56.560408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:56.561877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ed7bf-a70d-46de-aa23-a54d3b5a07f8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:56.568072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 658/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Jul 12 14:22:56 2022] GET /identity/v3/role_assignments?user.id=4a342fa16a564ef38675936d7f51578b&scope.project.id=35a95557d6a5450399d77fd04fed233c&role.id=640b98e8829b4de0bdceee05e74bf473 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:58.619182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43aa6-683f-4e28-ae05-d6e2adf7eb7b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:58.619778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43aa6-683f-4e28-ae05-d6e2adf7eb7b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:58.619778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a43aa6-683f-4e28-ae05-d6e2adf7eb7b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:58.621093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 630/1288] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:22:58.626150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:58.626546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:58.626706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:58.643645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:22:58.644848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:22:58.674061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:58.710815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 659/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:22:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:22:58.733398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43aa6-683f-4e28-ae05-d6e2adf7eb7b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:58.831889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a43aa6-683f-4e28-ae05-d6e2adf7eb7b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:58.842198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:22:58.865610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:58.867166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:58.867166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:58.867166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:58.867745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:58.868922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:22:58.880556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:22:58.881814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:22:58.882994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 631/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:22:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:22:58.887711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:58.983993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e8c66-6fe5-41cd-a148-96091e7e7cfa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:59.016868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:59.017558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:59.017845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:59.018122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:59.019359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:59.019669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:59.021036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a671709-16a0-4650-9816-330b0eb7cdff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:59.025811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 660/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 14:22:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:22:59.031708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:22:59.129095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d14b5-1eca-4de0-a3ee-1652aa78378b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:22:59.166091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:22:59.166738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:22:59.167040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:22:59.167306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:22:59.168034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:22:59.168319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:22:59.169889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88e69-83fc-47fd-821a-9024b5457b2a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:22:59.174804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 632/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 14:22:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:23:01.176107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92735-4cc3-4591-b27d-9c2a65e75b39 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:01.176660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92735-4cc3-4591-b27d-9c2a65e75b39 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:01.176660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92735-4cc3-4591-b27d-9c2a65e75b39 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:01.178068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 661/1293] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:23:01.183997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45942-8c75-4bde-9ef2-25c789a9b219 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:01.184447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45942-8c75-4bde-9ef2-25c789a9b219 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:01.184556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45942-8c75-4bde-9ef2-25c789a9b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:01.201939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e6b45942-8c75-4bde-9ef2-25c789a9b219 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:23:01.203459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e6b45942-8c75-4bde-9ef2-25c789a9b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:23:01.233608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:01.272404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 633/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:23:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:23:01.295373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd92735-4cc3-4591-b27d-9c2a65e75b39 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:01.325397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:01.396065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd92735-4cc3-4591-b27d-9c2a65e75b39 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:01.430817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:01.431416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:01.431696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:01.431969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:01.432676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:01.434062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5b486-8c31-4e18-9f75-01f0b4bde18b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:01.446922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 662/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Jul 12 14:23:01 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:23:03.512762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6912a-2ade-4640-b59e-0b98276ad24e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:03.513305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6912a-2ade-4640-b59e-0b98276ad24e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:03.513305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6912a-2ade-4640-b59e-0b98276ad24e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:03.514678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 634/1296] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:23:03.520265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:03.520695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:03.520822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:03.538004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:23:03.539197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:23:03.609092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 663/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:23:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:23:03.638122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6912a-2ade-4640-b59e-0b98276ad24e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:03.739661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6912a-2ade-4640-b59e-0b98276ad24e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:03.772242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:03.773017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:03.773299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:03.773621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:03.774264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:03.775619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:23:03.787269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:23:03.788409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:03.789794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 635/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:23:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:23:03.794522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:03.812978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:03.892609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f104a6-0170-4a38-90db-88058ae6978f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:03.925086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:03.925829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:03.926111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:03.926382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:03.927078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:23:03.927380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:03.928845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:03.934227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 664/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 14:23:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:23:03.940108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:03.972273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:04.038689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d4562-1f33-4eee-9681-172e791697ad None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:04.074346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:04.074987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:04.075273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:04.075554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:04.076337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] RBAC: Adding query filter params (interface=public, service_id=68da3d6611be49cfa1effa52e3310980, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:23:04.076649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:04.078032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7834be-7aa1-4eb0-bfc4-5767d493edec None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:04.083525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 636/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Jul 12 14:23:03 2022] GET /identity/v3/endpoints?service_id=68da3d6611be49cfa1effa52e3310980&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:23:04.089125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:04.181132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:04.202493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a5e52-87e9-4b44-8821-a2a527ea55ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:04.243256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:04.243922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:04.244205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:04.244482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:04.245161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:04.246571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3043f-fe5f-480c-abcd-4f567ef8c0f7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:04.253928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 665/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Jul 12 14:23:04 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:23:06.248419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5c1a-4946-4bc5-994f-abe23ba0a329 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:06.248976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5c1a-4946-4bc5-994f-abe23ba0a329 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:06.248976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5c1a-4946-4bc5-994f-abe23ba0a329 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:06.250353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 637/1302] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:23:06.256031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:06.256414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:06.256595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:06.278605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:23:06.280731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:23:06.346167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 666/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:23:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 14:23:06.369169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5c1a-4946-4bc5-994f-abe23ba0a329 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:06.472509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:06.479988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5c1a-4946-4bc5-994f-abe23ba0a329 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:06.513781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:06.514507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:06.514792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:06.515071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:06.515716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:06.517072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 14:23:06.528663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 14:23:06.529855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jul 12 14:23:06.530711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 638/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Jul 12 14:23:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 14:23:06.535944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:06.610246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:06.634489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae253763-6683-41d0-ab5f-7509e4f9b538 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:06.667514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:06.668181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:06.668532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:06.668746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:06.669485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:23:06.669792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:06.671163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104ee4-cf93-4ce3-8e42-dff340e2e821 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:06.677079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 667/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Jul 12 14:23:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:23:06.682928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:23:06.780490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f30d1c-5eab-41a8-b292-acb0fa6d60dd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:23:06.813572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:23:06.814214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:23:06.814498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:23:06.814776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:23:06.816067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:23:06.817451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d018ec2-b19b-4cbb-8545-900c6824c36f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:23:06.835025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 639/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Jul 12 14:23:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:23:16.673389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:23:16.831258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:09.683179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6eb49-50ca-42fc-bfba-1728dfb8557e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:09.683712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6eb49-50ca-42fc-bfba-1728dfb8557e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:09.683712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6eb49-50ca-42fc-bfba-1728dfb8557e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:09.688798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 668/1307] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:09.691062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40833-46ea-4dde-9195-8788d2b0377c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:09.691498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40833-46ea-4dde-9195-8788d2b0377c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:09.691679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40833-46ea-4dde-9195-8788d2b0377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:09.829267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9aa40833-46ea-4dde-9195-8788d2b0377c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:09.831151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9aa40833-46ea-4dde-9195-8788d2b0377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:10.001816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 640/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:10.057356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:10.057936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:10.058312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:10.059773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 669/1309] 10.222.0.25 () {54 vars in 964 bytes} [Tue Jul 12 14:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:10.069372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08bf99-2b01-4308-b8cf-3d22a7e69388 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:10.069372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08bf99-2b01-4308-b8cf-3d22a7e69388 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:10.070905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08bf99-2b01-4308-b8cf-3d22a7e69388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:10.150601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6d08bf99-2b01-4308-b8cf-3d22a7e69388 None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:10.151834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6d08bf99-2b01-4308-b8cf-3d22a7e69388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:10.174241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:10.233289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 641/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Jul 12 14:24:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:10.242157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:10.243896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:10.353656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572bf4-8fa7-4183-9527-c1947b34a7e2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:10.435126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:10.435780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:10.436062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:10.436339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:10.438015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:10.495700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:10.553880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:10.555754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e519e5-bbd1-4a92-bb0e-fc8afb07abef service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:10.575308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 670/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:12.871292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.871801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.871943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.873198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 642/1312] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:24:12.878565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd29d-0897-452e-b4a3-47323265f7df None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:12.878974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd29d-0897-452e-b4a3-47323265f7df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:12.879085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd29d-0897-452e-b4a3-47323265f7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:12.957957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-139bd29d-0897-452e-b4a3-47323265f7df None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:12.959219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-139bd29d-0897-452e-b4a3-47323265f7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:13.075457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 671/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:13.114996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:13.116524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:13.225536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c0f43-87ea-4b17-8491-21a8fe678b55 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:13.308560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:13.311223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:13.311862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:13.312104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:13.312386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:13.314505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:13.440677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:13.442135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90561e8b-4f2f-4992-a4f7-06f68a9651a2 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:13.460843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 643/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:24:15.826263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9891a-98c7-4d1f-a23f-ac264f30c9da None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.826775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9891a-98c7-4d1f-a23f-ac264f30c9da None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.826775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9891a-98c7-4d1f-a23f-ac264f30c9da None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.828109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 672/1315] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:15.833555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e815-72c2-4628-802c-e3969a80c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:15.834018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e815-72c2-4628-802c-e3969a80c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:15.834149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e815-72c2-4628-802c-e3969a80c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:15.898103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1c64e815-72c2-4628-802c-e3969a80c9d0 None None] MFA Rules not processed for user `9a6ac7756ddb4d96967796f1a0d107a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:15.899304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1c64e815-72c2-4628-802c-e3969a80c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:15.951054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:15.995144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 644/1316] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:16.050721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.051009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.051291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.052691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 673/1317] 10.222.0.25 () {54 vars in 964 bytes} [Tue Jul 12 14:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:16.058722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507b24a-cff5-41d8-a5a8-ca7ac5cc491e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.059115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507b24a-cff5-41d8-a5a8-ca7ac5cc491e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.059404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507b24a-cff5-41d8-a5a8-ca7ac5cc491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.133402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5507b24a-cff5-41d8-a5a8-ca7ac5cc491e None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:16.134603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5507b24a-cff5-41d8-a5a8-ca7ac5cc491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:16.210241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 645/1318] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Jul 12 14:24:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:16.219221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:16.221046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:16.231050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.324260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1b04-b1a6-4ae9-9ece-1804bbd6cd81 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:16.396200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:16.396775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:16.397065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:16.397357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:16.399015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:16.464706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:16.514490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:16.515925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497f76-461f-4471-8313-287b43a57cc0 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:16.537400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 674/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:19.445910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.446342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.446475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.447845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 646/1320] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:19.453110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e0de-a315-4801-9cb6-40b44d349059 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.453561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e0de-a315-4801-9cb6-40b44d349059 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.453688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9e0de-a315-4801-9cb6-40b44d349059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.515975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-17a9e0de-a315-4801-9cb6-40b44d349059 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:19.517184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17a9e0de-a315-4801-9cb6-40b44d349059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:19.611483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 675/1321] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:19.650260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:19.651785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:19.702010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:19.755119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db2c5b-05bc-4833-801b-a700750f18a9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:19.826493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:19.827309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:19.827596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:19.827875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:19.830158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:19.925177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:19.939601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:19.941024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1c4a-49d4-43bd-add9-860ebcc6e271 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:19.959692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 647/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:22.645191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:22.645734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:22.645850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:22.647153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 676/1323] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:22.652290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcaf0a-5e80-4018-b8cf-d61715eb4182 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:22.652710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcaf0a-5e80-4018-b8cf-d61715eb4182 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:22.652995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcaf0a-5e80-4018-b8cf-d61715eb4182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:22.717269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f7dcaf0a-5e80-4018-b8cf-d61715eb4182 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:22.718411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7dcaf0a-5e80-4018-b8cf-d61715eb4182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:22.823647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 648/1324] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:22.859772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:22.861937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:22.881898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:22.969823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231cae6-aca4-46d0-b0b0-ab60170600eb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:23.043414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:23.044064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:23.044345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:23.044620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:23.046474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:23.116633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:23.164730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:23.166131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58149f6-8551-4871-956b-ce515d6d5500 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:23.186396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 677/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:25.270951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.271464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.271464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.273147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 649/1326] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:25.278334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79229-127b-44c1-ad92-a66690860e54 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.278780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79229-127b-44c1-ad92-a66690860e54 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.278876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79229-127b-44c1-ad92-a66690860e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.343748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0db79229-127b-44c1-ad92-a66690860e54 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:25.345002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0db79229-127b-44c1-ad92-a66690860e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:25.449350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 678/1327] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:25.485084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:25.486800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:25.554461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:25.599240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30082a-1c8c-4602-a8ea-6bb4727740e4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:25.676014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:25.676663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:25.676949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:25.677230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:25.678939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:25.785147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:25.787748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:25.789755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e916e1f-e8a0-46ac-8687-f19f4a0e426a service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:25.817141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 650/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:27.889252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:27.889803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:27.889917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:27.891206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 679/1329] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:27.896422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edbc2-5b8c-45a7-b97f-6250c2725072 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:27.896824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edbc2-5b8c-45a7-b97f-6250c2725072 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:27.897166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edbc2-5b8c-45a7-b97f-6250c2725072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:27.965598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0e6edbc2-5b8c-45a7-b97f-6250c2725072 None None] MFA Rules not processed for user `9a6ac7756ddb4d96967796f1a0d107a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:27.966803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0e6edbc2-5b8c-45a7-b97f-6250c2725072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:28.067346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 651/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:28.105090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:28.106705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:28.137787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.211962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7247823-c281-45fa-aeb6-42c5cd355dc4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:28.286672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:28.287315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:28.287865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:28.288150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:28.289846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:28.365406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:28.419311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:28.420786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463d70-7029-439f-b055-49f4fcd17fef service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:28.443426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 680/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:30.707182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:30.707786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:30.707786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:30.709104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 652/1332] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:30.714152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94207a90-0a2d-4da8-a129-00209ad0e337 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:30.714600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94207a90-0a2d-4da8-a129-00209ad0e337 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:30.714794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94207a90-0a2d-4da8-a129-00209ad0e337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:30.778582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-94207a90-0a2d-4da8-a129-00209ad0e337 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:30.779787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-94207a90-0a2d-4da8-a129-00209ad0e337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:30.872313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 681/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:30.907789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:30.909353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:30.979110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:31.012737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256da6e-1226-4422-8cd9-b11e530408a8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:31.085093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:31.085765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:31.086047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:31.086327 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:31.087960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:31.195329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:31.196748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b719d-ea78-42bf-83c0-8aebe4673f83 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:31.207230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:31.216093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 653/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:33.276232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.276797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.276797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.278198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 682/1335] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:33.283792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cfbf-eff3-4a61-b91f-f8c85b00def1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.284209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cfbf-eff3-4a61-b91f-f8c85b00def1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.284341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cfbf-eff3-4a61-b91f-f8c85b00def1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.347448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4268cfbf-eff3-4a61-b91f-f8c85b00def1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:33.348624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4268cfbf-eff3-4a61-b91f-f8c85b00def1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:33.445297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 654/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:33.485217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:33.486841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:33.527257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.590168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775d44-aa2a-40b1-b43f-adb9817d5f29 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:33.662174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:33.662854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:33.663134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:33.663404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:33.665091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:33.746850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:33.772417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:33.773886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c023043-6594-4aed-bc56-04b3c2ab445e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:33.791344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 683/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:36.110491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.111058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.111058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.112428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 655/1338] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:36.117683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda3bc-bb9b-4100-ad25-2294fe7da1c8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.118073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda3bc-bb9b-4100-ad25-2294fe7da1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.118241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eda3bc-bb9b-4100-ad25-2294fe7da1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.182685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b4eda3bc-bb9b-4100-ad25-2294fe7da1c8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:36.183880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4eda3bc-bb9b-4100-ad25-2294fe7da1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:36.278170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 684/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:36.314763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:36.316333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:36.390772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:36.419401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139914-20a0-4896-8250-a7ed65a5ae9d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:36.492579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:36.493221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:36.493533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:36.493812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:36.495455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:36.601658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:36.603080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515311be-b798-4737-8656-ec498b16ae2b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:36.620500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 656/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:40.602255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:40.602808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:40.602808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:40.604154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 685/1341] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:40.609634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d47ffd-5279-4248-b532-a2371e3aeb8e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:40.610859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d47ffd-5279-4248-b532-a2371e3aeb8e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:40.610859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d47ffd-5279-4248-b532-a2371e3aeb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:40.614988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:40.676781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a4d47ffd-5279-4248-b532-a2371e3aeb8e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:40.678023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a4d47ffd-5279-4248-b532-a2371e3aeb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:40.776251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 657/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:40.814882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:40.816547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:40.866026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:40.923198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1152aca-724d-45f6-ba86-963bf6830b8b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:40.997341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:40.998024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:40.998310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:40.998620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:41.000294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:41.094032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:41.111761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:41.113210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce801beb-f0c5-4c0e-9a36-b22f936a48c7 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:41.131640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 686/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:43.839233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989439-33dd-4168-b644-eb9188648f41 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:43.839869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989439-33dd-4168-b644-eb9188648f41 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:43.839869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989439-33dd-4168-b644-eb9188648f41 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:43.841220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 658/1344] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:43.847225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebad88-a21d-44b3-9073-ed97da2ceaa5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:43.847655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebad88-a21d-44b3-9073-ed97da2ceaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:43.847778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebad88-a21d-44b3-9073-ed97da2ceaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:43.912787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-39ebad88-a21d-44b3-9073-ed97da2ceaa5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:43.914089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-39ebad88-a21d-44b3-9073-ed97da2ceaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:44.012146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 687/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:44.049960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989439-33dd-4168-b644-eb9188648f41 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:44.051444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-28989439-33dd-4168-b644-eb9188648f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:44.138327 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:44.157834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989439-33dd-4168-b644-eb9188648f41 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:44.235033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:44.235692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:44.235974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:44.236251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:44.237919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:44.350808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:44.352403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c9c0-8133-4e42-ac34-b27b354a1162 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:44.370464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 659/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:48.819696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:48.820223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:48.820319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:48.821677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 688/1347] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:24:48.826826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a252bc-1d3d-4c48-9171-9929150a5186 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:48.827240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a252bc-1d3d-4c48-9171-9929150a5186 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:48.827375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a252bc-1d3d-4c48-9171-9929150a5186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:48.843067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:48.896984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-79a252bc-1d3d-4c48-9171-9929150a5186 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:48.898218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79a252bc-1d3d-4c48-9171-9929150a5186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:48.996705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 660/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:49.034944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:24:49.036502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:49.096532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:49.142236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f928c7c-dd72-4ff5-9656-509f82361966 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:24:49.213058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:24:49.213733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:49.214011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:49.214282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:49.215912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:49.318285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:49.322541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:24:49.323961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099bd89-7758-49af-8f81-11c4101454cb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:24:49.342493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 689/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:24:54.266304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3296-c478-4ea6-b169-ecdf223506c7 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:54.267809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3296-c478-4ea6-b169-ecdf223506c7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:54.268313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3296-c478-4ea6-b169-ecdf223506c7 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:54.270051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 661/1350] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:24:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:54.275643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a09-5737-42ad-97f6-6ff1cf0f0b3e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:54.276043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a09-5737-42ad-97f6-6ff1cf0f0b3e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:54.276121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba4a09-5737-42ad-97f6-6ff1cf0f0b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:54.398827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-31ba4a09-5737-42ad-97f6-6ff1cf0f0b3e None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:54.400236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31ba4a09-5737-42ad-97f6-6ff1cf0f0b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:54.537283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 690/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:24:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:58.579174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b09882-a071-47fc-ad56-16a8304cd229 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:58.579732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b09882-a071-47fc-ad56-16a8304cd229 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:58.579958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b09882-a071-47fc-ad56-16a8304cd229 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:58.581609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 662/1352] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:24:58.586602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a37-ded1-415e-b787-59a3a64a59d4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:24:58.587008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a37-ded1-415e-b787-59a3a64a59d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:24:58.587244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd639a37-ded1-415e-b787-59a3a64a59d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:24:58.645724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:24:58.677428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd639a37-ded1-415e-b787-59a3a64a59d4 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:24:58.678539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd639a37-ded1-415e-b787-59a3a64a59d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:24:58.790086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 691/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:24:58.994568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=59 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:00.284358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a06a76-3855-453c-adf9-88590861c6d3 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:00.285097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a06a76-3855-453c-adf9-88590861c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:00.285097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a06a76-3855-453c-adf9-88590861c6d3 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:00.286861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 663/1354] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:00.292269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85260e-ea32-4363-bc41-e8f76107ddf4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:00.292739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85260e-ea32-4363-bc41-e8f76107ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:00.292821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85260e-ea32-4363-bc41-e8f76107ddf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:00.368035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7c85260e-ea32-4363-bc41-e8f76107ddf4 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:00.369274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7c85260e-ea32-4363-bc41-e8f76107ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:00.412408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:00.482230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 692/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:03.854991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483288a0-2dfd-4ed1-b464-c712579dce3a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:03.855603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483288a0-2dfd-4ed1-b464-c712579dce3a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:03.855603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483288a0-2dfd-4ed1-b464-c712579dce3a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:03.864411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 664/1356] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:03.867217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396973-c301-4065-8c2c-3d5fbe7df7cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:03.867217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396973-c301-4065-8c2c-3d5fbe7df7cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:03.867217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396973-c301-4065-8c2c-3d5fbe7df7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:03.959016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9c396973-c301-4065-8c2c-3d5fbe7df7cf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:03.960136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c396973-c301-4065-8c2c-3d5fbe7df7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:04.071922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 693/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:04.125705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:04.126368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:04.126889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:04.128664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 665/1358] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 14:25:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:04.136720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859326-57ba-4317-9d4e-22418c7b306e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:04.137005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859326-57ba-4317-9d4e-22418c7b306e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:04.137284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859326-57ba-4317-9d4e-22418c7b306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:04.162180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:04.230777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-da859326-57ba-4317-9d4e-22418c7b306e None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:04.232071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-da859326-57ba-4317-9d4e-22418c7b306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:04.370988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 694/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:04.379110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:04.380636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:04.539711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095f65f-eef6-4662-8a50-a18cdb06b4e8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:04.616562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:04.660204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:04.660942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:04.661229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:04.661529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:04.663150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:04.758695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:04.760280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c557f-ac30-475e-9e33-28d51cea89b7 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:04.779486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 666/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:25:07.018814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:07.019333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:07.019437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:07.021637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 695/1361] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:07.026355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49968-adc1-473b-8d53-8e20b12ee805 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:07.026872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49968-adc1-473b-8d53-8e20b12ee805 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:07.026952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49968-adc1-473b-8d53-8e20b12ee805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:07.094807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1ce49968-adc1-473b-8d53-8e20b12ee805 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:07.096002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1ce49968-adc1-473b-8d53-8e20b12ee805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:07.135039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:07.200482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 667/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:07.245841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:07.247822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:07.322232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:07.419163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69eab8-964e-4c74-abd2-623ac67d5e12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:07.557343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:07.558026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:07.558309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:07.558649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:07.560333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:07.584664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:07.656088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:07.657758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4995a73-601f-43b2-9d42-65a1da50b4ad service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:07.677766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 696/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:10.050971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106481df-06f9-4af9-999d-1fce29cae3bb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:10.051498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106481df-06f9-4af9-999d-1fce29cae3bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:10.051597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106481df-06f9-4af9-999d-1fce29cae3bb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:10.052888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 668/1364] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:10.058564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3deb7-9950-4699-bd5f-d7222f44585a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:10.059393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3deb7-9950-4699-bd5f-d7222f44585a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:10.059677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3deb7-9950-4699-bd5f-d7222f44585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:10.146698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1bc3deb7-9950-4699-bd5f-d7222f44585a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:10.147929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1bc3deb7-9950-4699-bd5f-d7222f44585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:10.251409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:10.257386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 697/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:10.300605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:10.300917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:10.301201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:10.303051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 669/1366] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 14:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:10.309054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e90dd-cf3f-4b41-bb82-93450254f8b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:10.309542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e90dd-cf3f-4b41-bb82-93450254f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:10.309923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e90dd-cf3f-4b41-bb82-93450254f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:10.386961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a68e90dd-cf3f-4b41-bb82-93450254f8b9 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:10.388170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a68e90dd-cf3f-4b41-bb82-93450254f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:10.519099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 698/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:10.526923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:10.528446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:10.685054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d22a13-f35d-4b22-8810-a3c058350f23 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:10.708624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:10.818147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:10.818799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:10.819085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:10.819488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:10.821194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:10.920879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:10.923189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628041e0-7e3f-4977-9886-80332cb93f48 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:10.942265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 670/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:11.981401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201de2-6bf9-46b7-8e4d-4cf3a31d7213 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:11.981955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201de2-6bf9-46b7-8e4d-4cf3a31d7213 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:11.981955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58201de2-6bf9-46b7-8e4d-4cf3a31d7213 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:11.983590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 699/1369] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:11.988780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7ddb-ca67-472d-b248-de5826807544 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:11.989207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7ddb-ca67-472d-b248-de5826807544 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:11.989415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d7ddb-ca67-472d-b248-de5826807544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:12.022499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:12.060220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-463d7ddb-ca67-472d-b248-de5826807544 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:12.061461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-463d7ddb-ca67-472d-b248-de5826807544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:12.166866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 671/1370] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:12.512288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:12.512580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:12.512862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:12.514529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 700/1371] 10.222.0.25 () {54 vars in 963 bytes} [Tue Jul 12 14:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:12.521221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe780fc-e0d6-4f78-b7f0-71e60e90093a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:12.521560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe780fc-e0d6-4f78-b7f0-71e60e90093a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:12.521877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe780fc-e0d6-4f78-b7f0-71e60e90093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:12.602267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dfe780fc-e0d6-4f78-b7f0-71e60e90093a None None] MFA Rules not processed for user `50b29bba6ee54cab9e3f6fdcc841ec77`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:12.603478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dfe780fc-e0d6-4f78-b7f0-71e60e90093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:12.652921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:12.684930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 672/1372] 10.222.0.25 () {58 vars in 1060 bytes} [Tue Jul 12 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:12.692390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:12.693948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:12.747555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:12.806997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b54fec-9c52-49a2-8198-0b40c5cfbccc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:12.883775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:12.884431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:12.884716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:12.885007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:12.886667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:12.992061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:13.005771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:13.007234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286d17e-1dbc-49d8-a665-5bb80a05ac29 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:13.028171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 701/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:13.047324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:13.047823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:13.047903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:13.053068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 673/1374] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:13.054805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35b589-c3b0-4ec0-929f-2baac7d5444c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:13.055235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35b589-c3b0-4ec0-929f-2baac7d5444c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:13.055845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35b589-c3b0-4ec0-929f-2baac7d5444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:13.123086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-da35b589-c3b0-4ec0-929f-2baac7d5444c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:13.124932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-da35b589-c3b0-4ec0-929f-2baac7d5444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:13.229879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 702/1375] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:13.271034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:13.273214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:13.467548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f9b56-6f52-48b3-bd18-e1cbe1118587 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:13.520326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:13.584248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:13.584886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:13.585165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:13.585473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:13.587123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:13.676716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:13.678373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0936e80-afdd-455a-94d5-35a0edecf288 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:13.697325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 674/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:15.762256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:15.762824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:15.763470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:15.766634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 703/1377] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:15.774195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021b1b-2a15-4e45-aec1-7bfdc17c54be None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:15.774667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021b1b-2a15-4e45-aec1-7bfdc17c54be None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:15.774771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021b1b-2a15-4e45-aec1-7bfdc17c54be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:15.843997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e3021b1b-2a15-4e45-aec1-7bfdc17c54be None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:15.845251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e3021b1b-2a15-4e45-aec1-7bfdc17c54be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:15.852984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:15.955275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 675/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:15.998887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:16.000963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:16.068942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:16.156708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aace1c2-86c8-4366-a2d2-15f501677bdd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:16.270013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:16.270664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:16.270951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:16.271229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:16.272935 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:16.292500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:16.364797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:16.366441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2162e7-4bf9-41b6-91f1-fe1b9c13a813 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:16.387212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 704/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:18.511157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:18.511684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:18.511684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:18.513051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 676/1380] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:18.518967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec103-8898-4070-bb6a-01b92ac7119d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:18.519400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec103-8898-4070-bb6a-01b92ac7119d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:18.519511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec103-8898-4070-bb6a-01b92ac7119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:18.591456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-055ec103-8898-4070-bb6a-01b92ac7119d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:18.593317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-055ec103-8898-4070-bb6a-01b92ac7119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:18.700373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:18.709380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 705/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:18.752090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:18.753850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:18.914775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:18.926514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e02f2-0aad-4275-b37e-4a9850917ffb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:19.051903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:19.052553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:19.052836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:19.053111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:19.054781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:19.147555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:19.149130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6665ef9-1b6c-4f0e-ad82-af73c7b945a6 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:19.161094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:19.171296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 677/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:21.427445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:21.427931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:21.428161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:21.430015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 706/1383] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:21.438209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78abc3-eeb3-44f8-b38c-d258d28d264a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:21.438599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78abc3-eeb3-44f8-b38c-d258d28d264a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:21.438944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78abc3-eeb3-44f8-b38c-d258d28d264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:21.505067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bc78abc3-eeb3-44f8-b38c-d258d28d264a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:21.506310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc78abc3-eeb3-44f8-b38c-d258d28d264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:21.605977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 678/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:21.648666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:21.650159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:21.799156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ea83-39b6-449c-b4e8-21165bcd921c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:21.868929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:21.924396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:21.925179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:21.925585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:21.925947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:21.928339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:22.048162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:22.049823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6825f-94d1-46f1-9b81-5b006962da37 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:22.068896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 707/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:25:24.096168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:24.096783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:24.096783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:24.098125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 679/1386] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:24.104789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e903-e42b-4bd1-bf5c-8d128b5dd1c4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:24.105212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e903-e42b-4bd1-bf5c-8d128b5dd1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:24.105617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e903-e42b-4bd1-bf5c-8d128b5dd1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:24.182907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ab75e903-e42b-4bd1-bf5c-8d128b5dd1c4 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:24.184144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab75e903-e42b-4bd1-bf5c-8d128b5dd1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:24.218633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:24.290111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 708/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:24.343471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:24.345003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:24.424211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:24.498638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef66fd1-3a5c-4279-869a-33f7e1fc1e5b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:24.643699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:24.644762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:24.645281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:24.645969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:24.648112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:24.692580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:24.746408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:24.748018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7093d4-910e-404d-a864-00dadc1a13ad service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:24.767133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 680/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:26.817247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:26.817856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:26.817856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:26.819162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 709/1389] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:26.824499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d26c-007f-4ae7-9576-00f09625f933 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:26.824935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d26c-007f-4ae7-9576-00f09625f933 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:26.825125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d26c-007f-4ae7-9576-00f09625f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:26.888979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c8d7d26c-007f-4ae7-9576-00f09625f933 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:26.890717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c8d7d26c-007f-4ae7-9576-00f09625f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:26.985937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 681/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:27.030340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:27.031873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:27.176826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e3cac-f86c-403f-8c92-170177e89ebd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:27.195983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:27.294538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:27.295195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:27.295480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:27.295756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:27.297479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:27.385922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:27.387529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad5faa-acd6-49ee-a754-32debec87ed0 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:27.407015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 710/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:28.116847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837794a-84e5-4cda-8f12-01fbc56b545d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:28.117485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837794a-84e5-4cda-8f12-01fbc56b545d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:28.117485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837794a-84e5-4cda-8f12-01fbc56b545d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:28.118792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 682/1392] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:28.124642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c5d3-2499-48d1-9281-06fb6683c662 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:28.125077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c5d3-2499-48d1-9281-06fb6683c662 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:28.125185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c5d3-2499-48d1-9281-06fb6683c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:28.151524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:28.194245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2415c5d3-2499-48d1-9281-06fb6683c662 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:28.195671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2415c5d3-2499-48d1-9281-06fb6683c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:28.291328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 711/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:28.346510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:28.346863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:28.347205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:28.348596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 683/1394] 10.222.0.25 () {54 vars in 962 bytes} [Tue Jul 12 14:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:28.353623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c947-40c5-4096-99a1-575cfeb19a67 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:28.354048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c947-40c5-4096-99a1-575cfeb19a67 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:28.354337 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c947-40c5-4096-99a1-575cfeb19a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:28.434847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1c74c947-40c5-4096-99a1-575cfeb19a67 None None] MFA Rules not processed for user `0df73889cb3f4bb58bdb5d674ed5379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:28.436111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1c74c947-40c5-4096-99a1-575cfeb19a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:28.476946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:28.512180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 712/1395] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Jul 12 14:25:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:28.520667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:28.522426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:28.538335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:28.628673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f9a0-ea10-4bd7-8f75-a914e87720fc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:28.703410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:28.703978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:28.704260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:28.704537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:28.706198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:28.772784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:28.814344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:28.815864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a6a9b8-39bb-4b6e-9bc7-6f5330bcc5a3 service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:28.835640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 684/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:29.460946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d8a7-69eb-4b3d-811f-97a655275b85 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.461549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d8a7-69eb-4b3d-811f-97a655275b85 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.461647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624d8a7-69eb-4b3d-811f-97a655275b85 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.463381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 713/1397] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:29.469311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.469792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.469898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.541742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:29.542967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:29.554267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086d130-23e5-4466-b514-e9686c6072d2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.554558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086d130-23e5-4466-b514-e9686c6072d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.554841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086d130-23e5-4466-b514-e9686c6072d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.620755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4086d130-23e5-4466-b514-e9686c6072d2 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:29.622149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4086d130-23e5-4466-b514-e9686c6072d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:29.641221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 685/1398] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:29.684528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:29.685770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:29.712810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:29.717627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 714/1399] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:25:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:29.764780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47a198-4876-48f4-95b4-5124e0cd4451 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.765215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47a198-4876-48f4-95b4-5124e0cd4451 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.765563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47a198-4876-48f4-95b4-5124e0cd4451 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.766934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 715/1400] 10.222.0.25 () {54 vars in 942 bytes} [Tue Jul 12 14:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:25:29.773173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.773581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.773862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.804931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7017c-e5b9-43c6-9d0a-80490b920c4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:29.843090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:29.851886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] MFA Rules not processed for user `9f47a2120e91452ebcaa6e042caed9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:29.853162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:29.921265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:29.921937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:29.922218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:29.922492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:29.924396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:29.953118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:29.961004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 716/1401] 10.222.0.25 () {58 vars in 1039 bytes} [Tue Jul 12 14:25:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:29.970991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:29.980311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfdf5c-1d66-426e-983a-8af8b6de3e67 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:30.030365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:30.031973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd611-945d-436b-af55-0615a5bda14b service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:30.050719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 686/1402] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:25:30.084163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:30.084814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:30.085092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:30.085410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:30.087096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:30.089792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:30.202510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:30.204188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988b935-dd0a-47f8-93c8-d33862833f5d service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:30.213483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 717/1403] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:25:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:25:32.172436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:32.173111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:32.173111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:32.178162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 687/1404] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:32.179603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e40b-9cb8-4c95-bc07-34a4543b9981 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:32.180016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e40b-9cb8-4c95-bc07-34a4543b9981 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:32.180171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e40b-9cb8-4c95-bc07-34a4543b9981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:32.250168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c132e40b-9cb8-4c95-bc07-34a4543b9981 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:32.251387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c132e40b-9cb8-4c95-bc07-34a4543b9981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:32.354066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:32.361152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 718/1405] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:32.406525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:32.407823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:32.573910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:32.577701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad756f-6d62-477d-990f-89b2ed2fadb9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:32.706984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:32.707751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:32.708102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:32.708447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:32.710229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:32.810459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:32.812125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da06ba-bfd0-4df5-80b6-ef673d9983fb service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:32.834231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 688/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:34.920327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:34.920908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:34.920908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:34.922286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 719/1407] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:34.927626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ee66-d92e-4eb3-bf0e-e30e733a20df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:34.928074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ee66-d92e-4eb3-bf0e-e30e733a20df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:34.928174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202ee66-d92e-4eb3-bf0e-e30e733a20df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:34.931696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:34.998481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9202ee66-d92e-4eb3-bf0e-e30e733a20df None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:34.999699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9202ee66-d92e-4eb3-bf0e-e30e733a20df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:35.103067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 689/1408] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:35.151246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:35.151246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:35.312862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76884b29-4eac-4478-b25b-32e252f373b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:35.389840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:35.444632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:35.445295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:35.445690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:35.445972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:35.447646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:35.567795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:35.570014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775502c-bea7-4c32-bf5e-2cabb6b4b361 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:35.622865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 720/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:45.101026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=64 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:45.606057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=64 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:52.244274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f0d2-9332-4602-aa4a-e964c0111fc1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:52.244839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f0d2-9332-4602-aa4a-e964c0111fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:52.244839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f0d2-9332-4602-aa4a-e964c0111fc1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:52.246250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 690/1410] 10.222.0.25 () {54 vars in 918 bytes} [Tue Jul 12 14:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:52.251955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35630d-3ac5-43bf-80da-f5a86ea278d5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:52.252385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35630d-3ac5-43bf-80da-f5a86ea278d5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:52.252505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35630d-3ac5-43bf-80da-f5a86ea278d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:52.326124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd35630d-3ac5-43bf-80da-f5a86ea278d5 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:52.327321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd35630d-3ac5-43bf-80da-f5a86ea278d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:52.443387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 721/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:52.457411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ab28-6ed4-42c7-bacb-0238816731d3 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:52.457702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ab28-6ed4-42c7-bacb-0238816731d3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:52.457980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834ab28-6ed4-42c7-bacb-0238816731d3 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:52.459328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 691/1412] 10.222.0.25 () {54 vars in 918 bytes} [Tue Jul 12 14:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:52.463766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d169-08a2-46cd-974b-c5ef6e044523 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:52.464051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d169-08a2-46cd-974b-c5ef6e044523 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:52.464324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2d169-08a2-46cd-974b-c5ef6e044523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:52.533538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:52.536116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-04e2d169-08a2-46cd-974b-c5ef6e044523 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:52.537933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-04e2d169-08a2-46cd-974b-c5ef6e044523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:52.651626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 722/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:55.128660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:55.129211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:55.129211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:55.130593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 692/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:55.135746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d95bb-7897-4c4f-a6ef-76aef2196dfa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:55.136181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d95bb-7897-4c4f-a6ef-76aef2196dfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:55.136316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d95bb-7897-4c4f-a6ef-76aef2196dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:55.201816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7f5d95bb-7897-4c4f-a6ef-76aef2196dfa None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:55.203025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f5d95bb-7897-4c4f-a6ef-76aef2196dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:55.257621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:55.299344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 723/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:55.342921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:55.344626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:55.494350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4c00-1eb3-4e9c-8293-8adaf886166a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:55.565977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:55.606559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:55.607945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:55.608223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:55.608496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:55.610157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:55.697615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:55.699138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9784b1f-9d83-4d0c-9bcd-3cb11e4533f4 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:55.717271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 693/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:56.622901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:56.623469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:56.623469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:56.624833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 724/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:56.629762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e654553-887c-4fee-af46-0596355e2cc7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:56.630158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e654553-887c-4fee-af46-0596355e2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:56.630356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e654553-887c-4fee-af46-0596355e2cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:56.703949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8e654553-887c-4fee-af46-0596355e2cc7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:56.705075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e654553-887c-4fee-af46-0596355e2cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:56.731816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:56.801395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 694/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:25:56.950851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:56.952378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:57.095563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212c43-218a-4580-a9d1-596998833e87 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:57.134004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:57.205808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:57.206452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:57.206731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:57.207000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:57.208654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:57.295180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:57.296752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f7888-9872-4bda-a91a-5d3a2a817fd1 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:57.315074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 725/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:58.181192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:58.181791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:58.181791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:58.183161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 695/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:58.188462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c97a8-9d48-4628-86e2-3e5a73925566 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:58.188910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c97a8-9d48-4628-86e2-3e5a73925566 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:58.189088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c97a8-9d48-4628-86e2-3e5a73925566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:58.242171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:58.265034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-576c97a8-9d48-4628-86e2-3e5a73925566 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:58.266273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-576c97a8-9d48-4628-86e2-3e5a73925566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:58.360898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 726/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:25:58.509288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:25:58.510915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:58.656593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83e756-d7b3-45c9-b7c1-9014a16d805b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:25:58.668284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:58.767748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:25:58.768316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:58.768631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:58.768911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:58.770600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:58.859321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:25:58.860942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576660-6150-4fd0-9804-c1ddb28fd4f3 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:25:58.878633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 696/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:25:59.730868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:59.731402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:59.731402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:59.733409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 727/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:25:59.738361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f8ea-41fd-4cc0-ab0c-8fcfdb702bb6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:25:59.738797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f8ea-41fd-4cc0-ab0c-8fcfdb702bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:25:59.738927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f8ea-41fd-4cc0-ab0c-8fcfdb702bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:25:59.757520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:25:59.813186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6f47f8ea-41fd-4cc0-ab0c-8fcfdb702bb6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:25:59.814431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6f47f8ea-41fd-4cc0-ab0c-8fcfdb702bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:25:59.908320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 697/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:26:00.057187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:00.058706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:00.170789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:00.207770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88852fef-eeca-46fe-88be-4079bc6d04ea None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:00.319874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:00.320443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:00.320729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:00.321012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:00.322702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:00.397407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:00.412144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:00.413791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498940b-51ba-4ef5-8809-3aa2f98e5209 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:00.432853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 728/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:01.291160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:01.291728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:01.291728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:01.293079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 698/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:01.298846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980fd24-86f5-48f1-b7dd-d0320683429a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:01.299229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980fd24-86f5-48f1-b7dd-d0320683429a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:01.299408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980fd24-86f5-48f1-b7dd-d0320683429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:01.378310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4980fd24-86f5-48f1-b7dd-d0320683429a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:01.379515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4980fd24-86f5-48f1-b7dd-d0320683429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:01.476028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 729/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:01.626463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:01.628022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:01.712031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:01.774072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c9cf6-4c6e-4615-a3da-f21e52af9cdf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:01.886328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:01.886897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:01.887184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:01.887462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:01.889672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:01.936755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:01.977067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:01.978693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49fd70-28bc-4192-b6c6-2996d380e9d8 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:01.996672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 699/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:02.798373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089cec0-5438-4959-858c-abf499f974bc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:02.798908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089cec0-5438-4959-858c-abf499f974bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:02.798908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089cec0-5438-4959-858c-abf499f974bc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:02.800241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 730/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:02.805581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd29b9-504c-452a-b144-a4966ed4bddb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:02.805985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd29b9-504c-452a-b144-a4966ed4bddb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:02.806152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd29b9-504c-452a-b144-a4966ed4bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:02.880286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c6dd29b9-504c-452a-b144-a4966ed4bddb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:02.881526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c6dd29b9-504c-452a-b144-a4966ed4bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:02.977510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 700/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:03.128132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089cec0-5438-4959-858c-abf499f974bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:03.129749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e089cec0-5438-4959-858c-abf499f974bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:03.177052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:03.274635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089cec0-5438-4959-858c-abf499f974bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:03.386576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:03.387186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:03.387467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:03.387867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:03.389634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:03.396821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:03.483546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:03.485156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2045f89-5740-45ed-b30e-184861d16566 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:03.503344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 731/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:04.299259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:04.299833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:04.299833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:04.301173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 701/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:04.306362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6e8b-5bd2-433e-ae89-8ab6001058a1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:04.306802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6e8b-5bd2-433e-ae89-8ab6001058a1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:04.306919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e6e8b-5bd2-433e-ae89-8ab6001058a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:04.381557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b09e6e8b-5bd2-433e-ae89-8ab6001058a1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:04.382760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b09e6e8b-5bd2-433e-ae89-8ab6001058a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:04.450478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:04.479310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 732/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:04.629514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:04.631133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:04.648850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:04.893663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767d564-61c8-4fe3-be69-3c6febf30c0c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:04.980572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:05.006558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:05.007198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:05.007486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:05.007767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:05.009437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:05.097126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:05.098732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d948b-b626-4e6a-bb1b-64304791b751 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:05.116241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 702/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:05.920364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:05.920926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:05.920926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:05.922495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 733/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:05.927639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0278-94e9-4d7f-983b-d2a2be9f9c85 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:05.928050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0278-94e9-4d7f-983b-d2a2be9f9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:05.928196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b0278-94e9-4d7f-983b-d2a2be9f9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:06.001534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-786b0278-94e9-4d7f-983b-d2a2be9f9c85 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:06.002738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-786b0278-94e9-4d7f-983b-d2a2be9f9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:06.042859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:06.101266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 703/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:06.251511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:06.253068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:06.398950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8d813-49f7-444f-934d-93ebd98f85b7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:06.453816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:06.511763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:06.512970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:06.513265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:06.513582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:06.515246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:06.602844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:06.604435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537ef87-f27c-4e1e-a70d-ea39edaba108 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:06.622871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 734/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:07.429533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:07.430066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:07.430165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:07.431768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 704/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:07.437145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb68e7-52a1-45c4-934e-612911197bda None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:07.437622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb68e7-52a1-45c4-934e-612911197bda None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:07.437732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb68e7-52a1-45c4-934e-612911197bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:07.510115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f3fb68e7-52a1-45c4-934e-612911197bda None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:07.511307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f3fb68e7-52a1-45c4-934e-612911197bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:07.513360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:07.607801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 735/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:07.756436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:07.757916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:07.903475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d154b-3af2-4880-947a-535858f3fc7e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:07.929695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:08.015288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:08.015856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:08.016138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:08.016414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:08.018101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:08.106672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:08.108433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bd838-9fbb-4a04-8a60-7f8382a865d5 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:08.126536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 705/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:10.040167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:10.040739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:10.040739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:10.042191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 736/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:10.048065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6974-7375-4f25-b5d6-4616b8070544 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:10.048517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6974-7375-4f25-b5d6-4616b8070544 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:10.048645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6974-7375-4f25-b5d6-4616b8070544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:10.087473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:10.115850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ce1d6974-7375-4f25-b5d6-4616b8070544 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:10.117054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ce1d6974-7375-4f25-b5d6-4616b8070544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:10.212918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 706/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:10.251368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:10.252917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:10.354826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ac680-aff6-4dff-9c3e-818256a89ac4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:10.386123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:10.437392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:10.438178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:10.438467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:10.438746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:10.440455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:10.547669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:10.549143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3cbb4-a26f-4897-85c0-ec9017b7df4f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:10.568303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 737/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:12.730854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:12.731934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:12.731934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:12.733244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 707/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:12.738562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b80c3-bd38-42f4-bac9-25d3f48b2193 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:12.738977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b80c3-bd38-42f4-bac9-25d3f48b2193 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:12.739115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b80c3-bd38-42f4-bac9-25d3f48b2193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:12.802313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a65b80c3-bd38-42f4-bac9-25d3f48b2193 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:12.803506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a65b80c3-bd38-42f4-bac9-25d3f48b2193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:12.816909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:12.900714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 738/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:12.939709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:12.941246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:13.042754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19f20e-dac4-47ee-ac95-94172628ae65 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:13.055096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:13.115676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:13.116312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:13.116591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:13.116888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:13.118541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:13.226772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:13.228186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455b45c-c0dd-47b1-919c-2be5032954cc service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:13.247177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 708/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:15.562342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:15.562900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:15.562900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:15.564339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 739/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:15.569458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6af790-9035-4d73-9865-41040170e04e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:15.569870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6af790-9035-4d73-9865-41040170e04e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:15.570025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6af790-9035-4d73-9865-41040170e04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:15.615731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:15.634068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6c6af790-9035-4d73-9865-41040170e04e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:15.635276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6c6af790-9035-4d73-9865-41040170e04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:15.729706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 709/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:15.768123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:15.769705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:15.873194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb479d81-e3b0-4de1-8d3f-7c323b64002e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:15.912219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:15.946865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:15.947507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:15.947789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:15.948061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:15.950414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:16.056254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:16.057636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf0253-ea41-4810-889c-e716189e70a7 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:16.076706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 740/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:18.203148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:18.203788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:18.203788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:18.205110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 710/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:18.210265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef9a1-9a46-4a2e-b738-2f77384e4ff3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:18.210710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef9a1-9a46-4a2e-b738-2f77384e4ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:18.210894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cef9a1-9a46-4a2e-b738-2f77384e4ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:18.274498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-61cef9a1-9a46-4a2e-b738-2f77384e4ff3 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:18.275711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61cef9a1-9a46-4a2e-b738-2f77384e4ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:18.296611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:18.372268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 741/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:18.409994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:18.411530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:18.516142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb67711-00d7-40f6-a5ce-425607e125f0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:18.535650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:18.589737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:18.590408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:18.590694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:18.590980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:18.592625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:18.705776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:18.707201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdc861-2fa5-4c27-910a-619d4dde1c0b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:18.725910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 711/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:20.830469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:20.830993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:20.831089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:20.832405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 742/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:20.837825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23d75-be59-4c96-8054-5785014b6619 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:20.838257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23d75-be59-4c96-8054-5785014b6619 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:20.838385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23d75-be59-4c96-8054-5785014b6619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:20.899069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:20.902191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-46b23d75-be59-4c96-8054-5785014b6619 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:20.903747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46b23d75-be59-4c96-8054-5785014b6619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:20.999376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 712/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:26:21.036531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:21.038112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:21.141913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877a85c-7bc8-41a1-8ab7-6a5e1ce0c466 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:21.191453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:21.217374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:21.218023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:21.218304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:21.218576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:21.220221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:21.328718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:21.330086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3eda19-409d-43cc-8c0a-52041d85cc9c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:21.347856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 743/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:23.395905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:23.396448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:23.396567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:23.397944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 713/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:23.403741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffd35-56ab-485d-9643-56e8a946ee50 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:23.404168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffd35-56ab-485d-9643-56e8a946ee50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:23.404326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffd35-56ab-485d-9643-56e8a946ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:23.470784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-636ffd35-56ab-485d-9643-56e8a946ee50 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:23.472080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-636ffd35-56ab-485d-9643-56e8a946ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:23.501814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:23.571817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 744/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:23.610117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:23.611667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:23.714860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e83de-dedb-4e1b-82f1-d2c8417fcffd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:23.742233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:23.786460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:23.787160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:23.787449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:23.787726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:23.789401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:23.896718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:23.898178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548a8f3-e6b8-44d2-bcfa-370599df9f76 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:23.916310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 714/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:25.976387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:25.976957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:25.976957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:25.978368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 745/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:26:25.984022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3d42-58aa-4abc-841a-afd10883e5b5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:25.984445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3d42-58aa-4abc-841a-afd10883e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:25.984580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc3d42-58aa-4abc-841a-afd10883e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:26.047420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ebdc3d42-58aa-4abc-841a-afd10883e5b5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:26.048622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ebdc3d42-58aa-4abc-841a-afd10883e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:26.057175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:26.145852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 715/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:26.182666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:26.184293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:26.287405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049ef26-9161-4b32-a36e-6346e30b9783 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:26.341789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:26.360313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:26.360963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:26.361251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:26.361567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:26.363216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:26.471398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:26.472822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972d98-93ec-4837-af80-4e6c125b57ce service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:26.490309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 746/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:28.559875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:28.560403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:28.560494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:28.561867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 716/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:28.567134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c252595-4c4d-41e2-ad29-38560b4da8f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:28.567580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c252595-4c4d-41e2-ad29-38560b4da8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:28.567685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c252595-4c4d-41e2-ad29-38560b4da8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:28.633662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5c252595-4c4d-41e2-ad29-38560b4da8f8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:28.634892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c252595-4c4d-41e2-ad29-38560b4da8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:28.673067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:28.730901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 747/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:28.768393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:28.770042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:28.872554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c1e63-334f-4a97-8766-58e1d17fc7bd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:28.911286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:28.947423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:28.947999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:28.948284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:28.948562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:28.951032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:29.057844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:29.059308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79f03c-0f12-474f-9efa-fe8457f22688 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:29.077498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 717/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:31.112292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:31.112857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:31.112857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:31.114269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 748/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:31.119541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa3335-8aa7-4d3a-8868-63493bc09ad1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:31.119941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa3335-8aa7-4d3a-8868-63493bc09ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:31.120096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa3335-8aa7-4d3a-8868-63493bc09ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:31.185077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-53aa3335-8aa7-4d3a-8868-63493bc09ad1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:31.186306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-53aa3335-8aa7-4d3a-8868-63493bc09ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:31.204467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:31.283637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 718/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:31.320859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:31.322437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:31.424547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c249c5-a9a4-4c55-9b83-fdfd015bddc4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:31.490498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:31.498360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:31.499003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:31.499283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:31.499558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:31.501191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:31.607212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:31.607862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa784e-52ce-4970-b21b-a8176d46f440 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:31.625999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 749/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:33.657781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:33.658314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:33.658314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:33.659674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 719/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:33.664836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4cd94-3f94-4bf7-a934-1de668d3702c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:33.665220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4cd94-3f94-4bf7-a934-1de668d3702c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:33.665441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4cd94-3f94-4bf7-a934-1de668d3702c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:33.732642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a6b4cd94-3f94-4bf7-a934-1de668d3702c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:33.733794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a6b4cd94-3f94-4bf7-a934-1de668d3702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:33.781404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:33.833052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 750/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:33.869097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:33.871288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:33.973819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257ccd-055a-4e17-b190-482d7c544d9d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:34.021197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:34.046872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:34.047506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:34.047787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:34.048063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:34.049733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:34.158321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:34.159749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c7fe-b2ee-458b-9344-7306a0e44ea4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:34.178660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 720/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:43.831297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=22 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:44.176327 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=66 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:53.921821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a080-d784-4f2c-b77b-0e8b7a8acbf1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:53.922395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a080-d784-4f2c-b77b-0e8b7a8acbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:53.922395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3a080-d784-4f2c-b77b-0e8b7a8acbf1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:53.923815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 751/1471] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:53.929012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b784b8-57da-4749-b46d-31b27154ca4b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:53.929506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b784b8-57da-4749-b46d-31b27154ca4b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:53.929612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b784b8-57da-4749-b46d-31b27154ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:53.992895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-39b784b8-57da-4749-b46d-31b27154ca4b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:53.994127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-39b784b8-57da-4749-b46d-31b27154ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:54.089569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 721/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:54.174302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:54.174656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:54.174974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:54.177080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 752/1473] 10.222.0.25 () {54 vars in 962 bytes} [Tue Jul 12 14:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:26:54.182414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd91f6-2072-4d6e-ad95-06ca93863001 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:54.183236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd91f6-2072-4d6e-ad95-06ca93863001 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:54.183565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd91f6-2072-4d6e-ad95-06ca93863001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:54.256720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ebcd91f6-2072-4d6e-ad95-06ca93863001 None None] MFA Rules not processed for user `0df73889cb3f4bb58bdb5d674ed5379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:54.257955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ebcd91f6-2072-4d6e-ad95-06ca93863001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:54.278894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:54.336827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 722/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Jul 12 14:26:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:54.344700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:54.346292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:54.447892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73c49-a90e-4d7f-8c39-11b53b8266b0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:54.520076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:54.520722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:54.521004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:54.521279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:54.522955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:54.579426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:54.633962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:54.635391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6e1dd-5e0a-4e6a-8361-541115322ae8 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:54.654462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 753/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:26:56.802646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:56.803184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:56.803184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:56.804499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 723/1476] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:56.810653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856b3b-b69f-4ede-b6a2-74d8ae7398b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:56.811100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856b3b-b69f-4ede-b6a2-74d8ae7398b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:56.811213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856b3b-b69f-4ede-b6a2-74d8ae7398b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:56.874524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9856b3b-b69f-4ede-b6a2-74d8ae7398b7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:56.875729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9856b3b-b69f-4ede-b6a2-74d8ae7398b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:56.977437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 754/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:57.002166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:57.003687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:57.128538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6998c-bbe4-4e34-8cf6-dba30073c50b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:57.175807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:57.220417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:57.221071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:57.221372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:57.221651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:57.222378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:57.224101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:26:57.224101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:26:57.224101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:26:57.225225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b64d20-e725-4ea2-814f-bca891726b9e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:57.243456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 724/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Jul 12 14:26:57 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:26:59.273498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:59.274062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:59.274062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:59.275584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 755/1479] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:26:59.280694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596107-54ce-481a-b99b-c2891857f06b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:59.281106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596107-54ce-481a-b99b-c2891857f06b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:59.281254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b596107-54ce-481a-b99b-c2891857f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:59.345798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2b596107-54ce-481a-b99b-c2891857f06b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:26:59.347026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2b596107-54ce-481a-b99b-c2891857f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:59.442595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 725/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:26:59.486300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:26:59.487845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:59.497625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:59.633129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328b2d3-bef3-4027-a6e5-3a1490b41eb3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:26:59.711582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:26:59.744018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:26:59.744670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:26:59.745525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:26:59.745805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:26:59.747474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:26:59.834272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:26:59.835883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03206527-178a-40f2-9970-6df240ee5f8e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:26:59.853732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 756/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:27:02.055838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:02.056430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:02.056430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:02.058339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 726/1482] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:27:02.063651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409f71-0c19-4544-958b-e86eed1eac10 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:02.064079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409f71-0c19-4544-958b-e86eed1eac10 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:02.064203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409f71-0c19-4544-958b-e86eed1eac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:02.127671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1f409f71-0c19-4544-958b-e86eed1eac10 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:27:02.128862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f409f71-0c19-4544-958b-e86eed1eac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:02.173813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:02.223903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 757/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:27:02.259718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:27:02.261893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:02.285646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:02.364517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb4bd9-a048-42af-ae34-437adad0d92f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:27:02.438380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:27:02.439026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:02.439307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:02.439582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:02.441206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:02.514392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:02.551054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:27:02.552485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d677-f25f-48dd-833c-2eb1691a02f0 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:27:02.570824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 727/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:27:04.650225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:04.651255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:04.651255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:04.652692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 758/1485] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:27:04.657789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c750685-6f0f-4556-a17f-72170de5202f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:04.658223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c750685-6f0f-4556-a17f-72170de5202f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:04.658384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c750685-6f0f-4556-a17f-72170de5202f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:04.723303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6c750685-6f0f-4556-a17f-72170de5202f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:27:04.724504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6c750685-6f0f-4556-a17f-72170de5202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:04.820498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 728/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:27:04.858873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:27:04.860428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:04.964459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712ccd8-ed52-45ac-beaf-0fb6885569a4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:27:05.153949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:05.155694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:27:05.156348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:05.156830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:05.157289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:05.159188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:05.265918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:27:05.267256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a09e6-847b-4a0a-b4cd-cc80afb18b98 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:27:05.285169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 759/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:27:07.404129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:07.404695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:07.404792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:07.406156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 729/1488] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:27:07.411530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcd062-3745-490b-b07b-76553846f0ee None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:07.411959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcd062-3745-490b-b07b-76553846f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:07.412090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcd062-3745-490b-b07b-76553846f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:07.475646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ffcd062-3745-490b-b07b-76553846f0ee None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:27:07.476851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ffcd062-3745-490b-b07b-76553846f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:07.528177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:07.571478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 760/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:27:07.612170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:27:07.613825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:07.651871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:07.719581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303e55-d00e-47b6-9926-da26cb3d4995 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:27:07.792099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:27:07.792747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:07.793031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:07.793306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:07.794983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:07.875568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:07.905297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:27:07.906748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085694f7-471d-4282-a6bc-7320f04bc355 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:27:07.924726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 730/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:27:10.200490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:10.201099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:10.201099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:10.202450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 761/1491] 10.222.0.25 () {54 vars in 926 bytes} [Tue Jul 12 14:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:27:10.207693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395102-0fad-4a64-b42d-f5d6171a567d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:10.208104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395102-0fad-4a64-b42d-f5d6171a567d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:10.208266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b395102-0fad-4a64-b42d-f5d6171a567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:10.273002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8b395102-0fad-4a64-b42d-f5d6171a567d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:27:10.274233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8b395102-0fad-4a64-b42d-f5d6171a567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:10.368860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 731/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Tue Jul 12 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:27:10.408311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:27:10.409904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:10.511652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eb340-d891-42bf-a318-550460fd6248 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:27:10.583983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:27:10.584623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:27:10.584903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:27:10.585176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:27:10.586841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:27:10.595208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:10.691799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:27:10.693140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0318f6-a747-4966-8130-e945f8707abf service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:27:10.711022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 762/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:27:20.367180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=82 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:27:20.708828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=50 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:36.847040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:36.847636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:36.847636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:36.920744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:28:36.921986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.017119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 732/1494] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:28:37.025743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.026236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.026341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.098287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:28:37.099487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.193524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 763/1495] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:28:37.197905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.199113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.210587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d0854-7a14-4ebd-a3e9-36ec8d62b9a1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:37.270829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:37.304960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:37.305893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.306172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.306445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.307156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:28:37.307460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:37.308462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:37.308462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:28:37.308462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:28:37.309361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:37.320721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 733/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:28:37.328782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.338668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563bf6d-8728-4c0a-b8a9-1f7cbdfcc211 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:37.399245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:37.435857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:37.436521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.436804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.437078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.437807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:37.439525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:37.439525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:28:37.439525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:28:37.440698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:37.495444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 764/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:28:37 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:28:37.499636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.501278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.534766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984abc-c598-433f-bca1-31017bcddaa5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:37.627086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:37.627747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.628037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.628313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.628986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:37.630022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:37.630022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:28:37.630022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:28:37.631218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:37.671093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 734/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:28:37 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:28:37.675253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.676837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.739137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706be2-af0d-4f07-85bc-25b8cc669110 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:37.820136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:37.831224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:37.831887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.832169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.832444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.833189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:37.834252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:37.834252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:28:37.834252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:28:37.835643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:37.847679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 765/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:28:37.852227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.853829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:37.865089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41880b9d-2b72-4381-ae90-da0df5180759 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:37.889658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:37.960183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:37.960842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:37.961123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:37.961429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:37.962148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:37.963263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:37.963263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:28:37.963263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:28:37.964147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c3a36-7c4d-4d55-956d-0035cdce0d97 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:37.976678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 735/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:28:37.980972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:37.982541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.015334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903e04-d169-431a-be2f-edf014bc8168 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:38.110957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:38.111840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.112114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.112381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] PATH_INFO: `/v3/projects/df4541fa66074bf5966ad870280cac1e/users/5786c8b626aa40b794f75e1fe3c1be89/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.134086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4541fa66074bf5966ad870280cac1e, user_id=5786c8b626aa40b794f75e1fe3c1be89, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:38.135149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:38.135149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:28:38.135149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:28:38.137068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709f843-b79b-49ad-b446-3851304a7ea1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:38.161020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 766/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:28:37 2022] PUT /identity/v3/projects/df4541fa66074bf5966ad870280cac1e/users/5786c8b626aa40b794f75e1fe3c1be89/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:28:38.173111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd943c-5bbe-4b71-8810-1dcc2e445de0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.173438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd943c-5bbe-4b71-8810-1dcc2e445de0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.173718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd943c-5bbe-4b71-8810-1dcc2e445de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.238458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-12bd943c-5bbe-4b71-8810-1dcc2e445de0 None None] MFA Rules not processed for user `5786c8b626aa40b794f75e1fe3c1be89`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:28:38.239678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-12bd943c-5bbe-4b71-8810-1dcc2e445de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.315793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 736/1502] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:28:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:28:38.328970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.329269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.329583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.330942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 767/1503] 10.222.0.25 () {54 vars in 963 bytes} [Tue Jul 12 14:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:28:38.337124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55047a-0014-42c9-b6b7-eb893ace1b21 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.337444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55047a-0014-42c9-b6b7-eb893ace1b21 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.337726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55047a-0014-42c9-b6b7-eb893ace1b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.368767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:38.412518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4b55047a-0014-42c9-b6b7-eb893ace1b21 None None] MFA Rules not processed for user `50b29bba6ee54cab9e3f6fdcc841ec77`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:28:38.413830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4b55047a-0014-42c9-b6b7-eb893ace1b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.488833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 737/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Tue Jul 12 14:28:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:28:38.496879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:38.498427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.548333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:38.601877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc659df7-cc44-4fa0-b189-84504de32780 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:38.674142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:38.674755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.675040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.675316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.677033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.781990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:38.783442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:38.786968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:38.802081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 768/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:28:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:28:38.844210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8bed-ab17-4eab-b89c-b6d0c941bb7a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:38.844495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8bed-ab17-4eab-b89c-b6d0c941bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:38.844776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8bed-ab17-4eab-b89c-b6d0c941bb7a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:38.846207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 738/1506] 10.222.0.25 () {52 vars in 829 bytes} [Tue Jul 12 14:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:28:38.849719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:38.851457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:38.954327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917bc58-5d9d-4fb8-af26-119ea545fa39 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:39.045916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:39.046781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:39.047057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:39.047352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] PATH_INFO: `/v3/users/5786c8b626aa40b794f75e1fe3c1be89` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:39.060181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5786c8b626aa40b794f75e1fe3c1be89)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:39.061196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:39.061196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:28:39.061196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:28:39.062432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:39.071973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:39.124865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5786c8b626aa40b794f75e1fe3c1be89'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:39.134364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5786c8b626aa40b794f75e1fe3c1be89'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:39.152724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5786c8b626aa40b794f75e1fe3c1be89'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:39.153190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5786c8b626aa40b794f75e1fe3c1be89'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:39.164868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5786c8b626aa40b794f75e1fe3c1be89'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:39.170547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 769/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:28:38 2022] DELETE /identity/v3/users/5786c8b626aa40b794f75e1fe3c1be89 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:28:39.175016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:39.175580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:39.175580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:39.231701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:28:39.232824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:39.307030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:39.319843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 739/1508] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:28:39.329756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:39.331156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:39.432861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73212b3-68f1-4eef-b179-96acf4f5c0e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:39.486465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:39.507487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:39.508141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:39.508421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:39.508697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:39.510378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:39.617596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:39.619031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b4ca3-0139-4470-9cfd-4ad279e27bc6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:39.637572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 770/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:28:39.979369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:28:39.981171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:28:39.992625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06838d-c282-468c-96dc-bad2aa1c236f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:28:40.095968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:28:40.096826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:28:40.097107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:28:40.097410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] PATH_INFO: `/v3/projects/df4541fa66074bf5966ad870280cac1e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:28:40.107961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4541fa66074bf5966ad870280cac1e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:28:40.109109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:28:40.109109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:28:40.109109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:28:40.110228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:28:40.167529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4541fa66074bf5966ad870280cac1e', 'request_id': 'req-112ffdd1-9f74-47d9-b0e0-f081443e8468'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:40.177114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4541fa66074bf5966ad870280cac1e', 'request_id': 'req-112ffdd1-9f74-47d9-b0e0-f081443e8468'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:40.184516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4541fa66074bf5966ad870280cac1e', 'request_id': 'req-112ffdd1-9f74-47d9-b0e0-f081443e8468'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:40.215583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] The token cache is being invalidate because project df4541fa66074bf5966ad870280cac1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:28:40.215867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:40.216296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-112ffdd1-9f74-47d9-b0e0-f081443e8468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:28:40.217767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 740/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:28:39 2022] DELETE /identity/v3/projects/df4541fa66074bf5966ad870280cac1e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:28:49.635261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=62 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:28:50.213654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:20.817092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:20.817689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:20.817689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:20.818979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 771/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:20.823991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb071cd-9c8d-4398-95bf-6eb7dab3d6a2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:20.824460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb071cd-9c8d-4398-95bf-6eb7dab3d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:20.824568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb071cd-9c8d-4398-95bf-6eb7dab3d6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:20.904917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3eb071cd-9c8d-4398-95bf-6eb7dab3d6a2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:20.906154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3eb071cd-9c8d-4398-95bf-6eb7dab3d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:21.014261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 741/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:30:21.169883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:21.172023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:21.289466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d87a7f-54ea-4c70-afaa-4f74ccbaa9d6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:21.294948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:21.295630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:21.295920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:21.296203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:21.304620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:21.306499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4df017-621c-4801-b163-9dbc73b176cb service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:21.316124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 772/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:22.143162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93124124-cb3a-4c84-8329-6f0ea217fe32 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:22.143720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93124124-cb3a-4c84-8329-6f0ea217fe32 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:22.143720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93124124-cb3a-4c84-8329-6f0ea217fe32 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:22.145638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 742/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:22.150512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae788d-65d9-4e2f-ad89-479e16f869c3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:22.150925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae788d-65d9-4e2f-ad89-479e16f869c3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:22.151058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae788d-65d9-4e2f-ad89-479e16f869c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:22.169629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d8ae788d-65d9-4e2f-ad89-479e16f869c3 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:22.170824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d8ae788d-65d9-4e2f-ad89-479e16f869c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:22.179117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 773/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:30:22.329069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93124124-cb3a-4c84-8329-6f0ea217fe32 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:22.343546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93124124-cb3a-4c84-8329-6f0ea217fe32 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:22.349464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:22.350092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:22.350376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:22.350654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:22.358895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:22.360461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a0f79-54b8-4c04-b866-0a4df2cfa622 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:22.370825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 743/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:23.546188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd5f1-4600-43ec-bfb5-5694030b5c53 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:23.546615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd5f1-4600-43ec-bfb5-5694030b5c53 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:23.546769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd5f1-4600-43ec-bfb5-5694030b5c53 None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:23.548160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 774/1517] 10.222.0.76 () {54 vars in 925 bytes} [Tue Jul 12 14:30:23 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:30:23.557476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbe9e-d07b-4616-981d-500147e589e0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:23.557906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbe9e-d07b-4616-981d-500147e589e0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:23.558040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cbe9e-d07b-4616-981d-500147e589e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:23.616383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c5cbe9e-d07b-4616-981d-500147e589e0 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:23.617623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c5cbe9e-d07b-4616-981d-500147e589e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:23.635534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:23.690220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 744/1518] 10.222.0.76 () {58 vars in 1015 bytes} [Tue Jul 12 14:30:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:32.164949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=56 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:33.345818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670467bd-baa6-49d0-a663-cec43fbba22a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.346470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670467bd-baa6-49d0-a663-cec43fbba22a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.346470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670467bd-baa6-49d0-a663-cec43fbba22a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.347804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 775/1519] 10.222.0.25 () {54 vars in 960 bytes} [Tue Jul 12 14:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:33.352876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca992f2c-6466-4403-b546-17d73e1359d6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.353321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca992f2c-6466-4403-b546-17d73e1359d6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.353458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca992f2c-6466-4403-b546-17d73e1359d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.373976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ca992f2c-6466-4403-b546-17d73e1359d6 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:33.375228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ca992f2c-6466-4403-b546-17d73e1359d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:33.384369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 745/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Tue Jul 12 14:30:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:33.391927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670467bd-baa6-49d0-a663-cec43fbba22a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:33.405470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670467bd-baa6-49d0-a663-cec43fbba22a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:33.410403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:33.411048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.411333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.411610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.420556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:33.422288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969b600-42f9-4032-bcfe-b66e574202de service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:33.433487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 776/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:33.609758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42a3f-efe4-4f30-bf1c-827b2d356252 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.609960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42a3f-efe4-4f30-bf1c-827b2d356252 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.610247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42a3f-efe4-4f30-bf1c-827b2d356252 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.611622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 746/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:33.616980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caac65b-cdaa-4c52-b9bc-1dbb0a0f3461 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.617268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caac65b-cdaa-4c52-b9bc-1dbb0a0f3461 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.617580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caac65b-cdaa-4c52-b9bc-1dbb0a0f3461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.644689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5caac65b-cdaa-4c52-b9bc-1dbb0a0f3461 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:33.645911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5caac65b-cdaa-4c52-b9bc-1dbb0a0f3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:33.654417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 777/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:33.809855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd42a3f-efe4-4f30-bf1c-827b2d356252 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:33.824463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd42a3f-efe4-4f30-bf1c-827b2d356252 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:33.829918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:33.830557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:33.830844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:33.831125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:33.839947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:33.841554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07faad95-4b56-4b90-9918-cd76140165a9 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:33.851502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 747/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:34.683453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198b1e-c333-47a2-b554-daa98b411a1d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:34.683979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198b1e-c333-47a2-b554-daa98b411a1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:34.683979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50198b1e-c333-47a2-b554-daa98b411a1d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:34.685362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 778/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 14:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:34.689995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a985-b2d4-47ae-a2aa-22121df60338 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:34.690394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a985-b2d4-47ae-a2aa-22121df60338 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:34.690548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3a985-b2d4-47ae-a2aa-22121df60338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:34.711079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-79d3a985-b2d4-47ae-a2aa-22121df60338 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:34.712457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79d3a985-b2d4-47ae-a2aa-22121df60338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:34.721254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 748/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:34.879942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50198b1e-c333-47a2-b554-daa98b411a1d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:34.894177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50198b1e-c333-47a2-b554-daa98b411a1d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:34.899542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:34.900178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:34.900459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:34.900738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:34.908787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:34.910401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead5279-e91c-4056-8146-708ab12ce59d service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:34.919702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 779/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:38.959235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:38.959799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:38.959799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:38.961303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 749/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:38.966434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3231544-0f9d-4a7a-b5d3-811ca527aee9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:38.966860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3231544-0f9d-4a7a-b5d3-811ca527aee9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:38.966990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3231544-0f9d-4a7a-b5d3-811ca527aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:38.985861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c3231544-0f9d-4a7a-b5d3-811ca527aee9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:38.987059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c3231544-0f9d-4a7a-b5d3-811ca527aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:38.995495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 780/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:39.068626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:39.071468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:39.154346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34439a4f-b3c4-409b-9219-62b783319b4f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:39.157844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:39.158512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:39.158819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:39.159101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:39.168807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:39.170344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5f507-e186-4b30-a531-4280048c5e0c service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:39.180217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 750/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:41.286696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971ecc-3c8d-4b54-b81e-2cacfbab5019 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:41.287308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971ecc-3c8d-4b54-b81e-2cacfbab5019 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:41.287308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971ecc-3c8d-4b54-b81e-2cacfbab5019 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:41.288678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 781/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:41.294063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafff3-ac92-4399-b73b-3877b79c49dc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:41.294509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafff3-ac92-4399-b73b-3877b79c49dc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:41.294620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbafff3-ac92-4399-b73b-3877b79c49dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:41.313147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6dbafff3-ac92-4399-b73b-3877b79c49dc None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:41.314361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6dbafff3-ac92-4399-b73b-3877b79c49dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:41.322270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 751/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:41.361132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71971ecc-3c8d-4b54-b81e-2cacfbab5019 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:41.372089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71971ecc-3c8d-4b54-b81e-2cacfbab5019 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:41.375811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:41.376440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:41.376723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:41.377003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:41.386160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:41.387628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb328c-edf9-47b2-98fb-b1da691dfc14 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:41.398003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 782/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:45.030849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:45.031409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:45.031409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:45.032767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 752/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:45.037909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5445f-ca70-4285-bf2b-728c4d20cbbf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:45.038372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5445f-ca70-4285-bf2b-728c4d20cbbf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:45.038501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5445f-ca70-4285-bf2b-728c4d20cbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:45.056242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f2a5445f-ca70-4285-bf2b-728c4d20cbbf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:45.057737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f2a5445f-ca70-4285-bf2b-728c4d20cbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:45.065497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 783/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:45.368298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:45.369980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:45.434524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:45.448604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73016877-03d7-4011-9fa8-f22c65b62610 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:45.452140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:45.452775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:45.453058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:45.453361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:45.462725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:45.464142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67e883-9e3d-4070-a702-49dfd314f9ba service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:45.473671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 753/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:47.558523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:47.559043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:47.559138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:47.560621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 784/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:30:47.574200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005360-2bca-4c47-91eb-ffa34a398b8f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:47.574723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005360-2bca-4c47-91eb-ffa34a398b8f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:47.574842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92005360-2bca-4c47-91eb-ffa34a398b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:47.594497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-92005360-2bca-4c47-91eb-ffa34a398b8f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:30:47.595682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-92005360-2bca-4c47-91eb-ffa34a398b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:47.604967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 754/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:30:47.921152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:30:47.922863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:30:47.937771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2411907-9c3c-4d30-a04a-c882e80fd8af None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:30:47.941848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:30:47.942489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:30:47.942768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:30:47.943041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:30:47.952684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:30:47.954356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593bb7c-20b1-4e3b-b856-341592ad2ae7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:30:47.964920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 785/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:30:57.588963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=11 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:30:57.960222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=33 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:17.720110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bfc56-8505-4aba-8e35-7bd8ebbfb239 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.721221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bfc56-8505-4aba-8e35-7bd8ebbfb239 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.721755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bfc56-8505-4aba-8e35-7bd8ebbfb239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.724701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f184fd-6661-4fdb-bff6-c339b4cc2180 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.725707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f184fd-6661-4fdb-bff6-c339b4cc2180 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.726221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f184fd-6661-4fdb-bff6-c339b4cc2180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.760391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-736bfc56-8505-4aba-8e35-7bd8ebbfb239 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.761620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-736bfc56-8505-4aba-8e35-7bd8ebbfb239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.762467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-29f184fd-6661-4fdb-bff6-c339b4cc2180 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.763649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-29f184fd-6661-4fdb-bff6-c339b4cc2180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.778707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 755/1540] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.779951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672d2-fddb-4c9c-95db-dbeba40fdb02 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.780079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 786/1541] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.780177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672d2-fddb-4c9c-95db-dbeba40fdb02 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.780404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901672d2-fddb-4c9c-95db-dbeba40fdb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.794730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea35a-60c2-490a-985a-df4a1d2b9b6e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.795028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea35a-60c2-490a-985a-df4a1d2b9b6e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.795284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea35a-60c2-490a-985a-df4a1d2b9b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.810473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-901672d2-fddb-4c9c-95db-dbeba40fdb02 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.811681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-901672d2-fddb-4c9c-95db-dbeba40fdb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.824078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 756/1542] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.826082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6874ba-9a1e-479a-80af-4111c04e1bfd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.826379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6874ba-9a1e-479a-80af-4111c04e1bfd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.826657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6874ba-9a1e-479a-80af-4111c04e1bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.834405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-851ea35a-60c2-490a-985a-df4a1d2b9b6e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.835616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-851ea35a-60c2-490a-985a-df4a1d2b9b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.859279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 787/1543] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:17.860608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.861541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.861541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.862627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ef6874ba-9a1e-479a-80af-4111c04e1bfd None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.863970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ef6874ba-9a1e-479a-80af-4111c04e1bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.885766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 757/1544] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.886844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.887130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.887406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.898361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.899801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.915545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 788/1545] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.916225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:17.927774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:17.929133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:17.936807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ef5a2-ba5d-45c1-82af-86e1e37372bb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:17.949820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:17.950497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.950781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.951057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.951720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 758/1546] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:17.952567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:17.954236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:17.954793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:17.956049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:17.956049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:32:17.956049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:32:17.957349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:17.966887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5a78-bcc2-4e0c-ade5-5ec62b556b61 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:17.972439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:17.973045 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:17.973349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:17.973649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:17.974385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:17.974701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:17.976308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78713d30-c0ad-405c-b817-c5abb5fb01a3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:17.977296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 789/1547] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:17.978402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:17.995785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c2a74-64be-4ae4-a2d1-969cafa91067 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:17.997975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 759/1548] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.000275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588c35-32ec-41b4-af8e-d8904f0ccaf7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.000761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588c35-32ec-41b4-af8e-d8904f0ccaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.001231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b588c35-32ec-41b4-af8e-d8904f0ccaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.001906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.002991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.003438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.003867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.004766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.005262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.006988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8555d-e5f0-404a-bf58-7f02746a1f6a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.021276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7b588c35-32ec-41b4-af8e-d8904f0ccaf7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.021568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 790/1549] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.022631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.023212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7b588c35-32ec-41b4-af8e-d8904f0ccaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.023333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.023558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.036630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 760/1550] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.038325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3e51-79ab-4d22-985f-9bb57e99dc1c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.038825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3e51-79ab-4d22-985f-9bb57e99dc1c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.039314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3e51-79ab-4d22-985f-9bb57e99dc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.046958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.048466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.058227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 791/1551] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.060143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-199f3e51-79ab-4d22-985f-9bb57e99dc1c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.061375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-199f3e51-79ab-4d22-985f-9bb57e99dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.062485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.070543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 761/1552] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.071921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.072214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.072492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.074249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50da232-08e4-4f03-a75c-78735478f4b5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.078834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.079998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.080278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.080551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.081415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.083294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.092207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.093442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.101476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 762/1553] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.102230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.112723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eeeee-8a66-4b7e-b861-ac5a1f6be167 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.118321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.118998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.119282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.119561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.120245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.121260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:18.121260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:32:18.121260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:32:18.125368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 792/1554] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.125519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a7d40-96df-47c6-87f5-9212f13acd86 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.125628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.150207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 763/1555] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.151553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1a49-e1c5-47a7-bb08-bcd362adb477 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.151848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1a49-e1c5-47a7-bb08-bcd362adb477 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.152129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1a49-e1c5-47a7-bb08-bcd362adb477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.169451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4ded1a49-e1c5-47a7-bb08-bcd362adb477 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.170669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4ded1a49-e1c5-47a7-bb08-bcd362adb477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.186851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a844a4-f8b4-4eab-8864-b4c1bd910677 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.237716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 764/1556] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.238969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced9ca8-8361-4d42-a22c-6fba7ccd06f1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.239255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced9ca8-8361-4d42-a22c-6fba7ccd06f1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.239636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced9ca8-8361-4d42-a22c-6fba7ccd06f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.251717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.252380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.252658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.252934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.253651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.255553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.257689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aced9ca8-8361-4d42-a22c-6fba7ccd06f1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.258882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aced9ca8-8361-4d42-a22c-6fba7ccd06f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.267380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 765/1557] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.268435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340649-30b6-4010-975d-cc99aa2a75f0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.268720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340649-30b6-4010-975d-cc99aa2a75f0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.268997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf340649-30b6-4010-975d-cc99aa2a75f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.285323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 793/1558] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.286262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.286841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bf340649-30b6-4010-975d-cc99aa2a75f0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.288066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf340649-30b6-4010-975d-cc99aa2a75f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.296146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:18.348213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 766/1559] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.349669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.349813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f56c25-4583-4b90-9445-f1be29751c99 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.350009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.350247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.354979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.355645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.355928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.356205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.356934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.357240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.358877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.368280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.369522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.370830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 794/1560] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.371612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.381669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 767/1561] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.382766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.388368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23e1ff-e1dd-421c-8cf1-ba05d9a34f61 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.393082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a6ea8-3d25-41af-8cd4-886e9182c9ce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.393833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.394525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.394811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.395093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.395855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.396909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:18.396909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:32:18.396909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:32:18.397772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.398143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.398402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.398683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.398955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.399626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.401527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.408827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:18.425417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 795/1562] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 285 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.426288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.426506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 768/1563] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.427323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.437132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a48c-33ac-4520-9bdf-59954062367c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.441167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12041ad1-bc1f-4915-bfb3-3b5a05c05565 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.442274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.442943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.443187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.443461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.444174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.444478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.445862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.446107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.446510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.446793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.447071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.447790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.448105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.449709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c32bc-c983-42eb-9b84-071f57f23290 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.458036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 769/1564] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.458795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.461323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 796/1565] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.462646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.462936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.463217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.475132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f9d98-5958-474a-93fe-78291c585098 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.479703 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.480359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.480641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.480914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.481632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.482244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:18.483475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:18.483572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.492076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 797/1566] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:18.492763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.502832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb206de-2ebf-4c0b-b12a-839901e2d4ab None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.505183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 770/1567] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.506032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.507542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.508431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.508727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.509006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.509731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.511640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.516257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614c2bb-760f-4417-860f-d40ce3f501de admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.520885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.521553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.521836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.522124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.522864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.523167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.524737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.540678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 798/1568] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.540889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 771/1569] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.540889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.545227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.548862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b0314-9047-48cf-9bfc-57f613f1c268 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.605110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.605805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.606090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.606366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.607078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.608656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.609847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e07d7-6f42-4039-ab59-b5ff180e02da admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.615679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.616385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.616669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.616946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.617700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.622182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.622346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 772/1570] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.622346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.630176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f7709-807c-475b-b39d-8b20d6bf1d0b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.635208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 799/1571] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.635208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.637637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.638273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.638554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.638828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.639503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.641488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.642783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d483a9-c4aa-4d04-8815-06e2fa37c3a0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.647317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.647962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.648247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.648524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.649197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.651098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.667474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 773/1572] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.668275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.682214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 800/1573] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.682214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.700911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:18.730197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf410eb-8bad-4159-b2c9-a6ed986dfa0e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.745813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e03189-3b45-45da-ab15-87481e6c7819 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.750374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.751060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.751343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.751621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.752339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.753999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.755652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:18.764629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 801/1574] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.765521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.775063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0b358-02e9-44ce-a89b-fa06c826454e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.779969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.781250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.781568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.781851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.783015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.784983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.789984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.790645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.790928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.791204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.791924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:18.792228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.793864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.805540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 774/1575] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:18.809525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 802/1576] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.809652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.809773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.818869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aaaab-8d3d-4ee9-806d-c2d0115cf8a0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.819240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876653-3acd-4d0b-a2a2-c9848764f257 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.823452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.823935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.824092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.824380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.824656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.824769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.825022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.825301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.825428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.826036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.826965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.827885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.836921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 803/1577] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.837810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.848109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996e6b-43d1-4fec-bf1b-fd123b1ffd0f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.852922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.856863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 775/1578] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.857013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.857013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.857013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.857222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.857311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.857311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.867710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 804/1579] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.868480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.883168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59c11f-52e5-4298-9ee0-9adc4500a5ee admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.919902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62594-c635-4cb8-b76e-ec06f82b9462 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.925149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.925844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.926126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.926397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.927078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.928977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.941479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.942159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.942439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.942714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.943397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.945411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.950681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 776/1580] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.951307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.961760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fd155-9fbc-4b37-afce-4c419282100f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.969402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 805/1581] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:18.969550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.969550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.969763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.969861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.969861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.969861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.970075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.976541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85722ee-f7e2-4d9b-b59a-e486b4ffc39a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.980486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 777/1582] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.981289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.982063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.982416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.982528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.982806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:18.983646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:18.985780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:18.993042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b46c3-d6ae-4b1d-bc0c-6dc90432598b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:18.995726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 806/1583] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:18.996549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:18.998658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:18.999163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:18.999444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:18.999722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.000434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.003713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.006288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0655c-625e-4071-89f6-be9d105033ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.010807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.011665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.011947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.012222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] PATH_INFO: `/v3/projects/eb73ce3627964df4a085d7d9a81a9f59/users/b22129ca8c714db6ad44dee94e6099e9/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.027315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb73ce3627964df4a085d7d9a81a9f59, user_id=b22129ca8c714db6ad44dee94e6099e9, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.029837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 778/1584] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:18 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:19.030685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.033069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.046999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 807/1585] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:18 2022] PUT /identity/v3/projects/eb73ce3627964df4a085d7d9a81a9f59/users/b22129ca8c714db6ad44dee94e6099e9/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.048519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.080393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.095632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.103586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b88f4-7bc3-4564-8272-4b24516c9616 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.120830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6891-77bb-4882-9837-b7791747534b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.126333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.127234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.127536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.127813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] PATH_INFO: `/v3/projects/2db99ccb0a8845a48cba15c209b8a425/users/3791cb0aef234ac994507815453f7b63/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.142632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db99ccb0a8845a48cba15c209b8a425, user_id=3791cb0aef234ac994507815453f7b63, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.145312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.158863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 808/1586] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:19 2022] PUT /identity/v3/projects/2db99ccb0a8845a48cba15c209b8a425/users/3791cb0aef234ac994507815453f7b63/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.159656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.162719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.163584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.163870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.164143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.164833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.166772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.189206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 779/1587] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:19.190075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.200524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f019a65-b17c-4aa0-aab9-db74f90d9c35 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.229986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bcbe1-da8e-40df-85fb-287b7690cfbd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.234697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.235414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.235700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.235981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.236711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.238343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.248620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 809/1588] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.249463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.261243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6b42e-54b6-4666-8b91-262aa95a66c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.264356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65659929-becd-4212-a074-902f978de64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.265029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65659929-becd-4212-a074-902f978de64a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.265318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65659929-becd-4212-a074-902f978de64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.265622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65659929-becd-4212-a074-902f978de64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.266108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.266888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65659929-becd-4212-a074-902f978de64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.267600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.267882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.268162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] PATH_INFO: `/v3/projects/bd2e10a0973e431db6cc11cc924fd826/users/d1ac1cdd046a4ae4b9f41b75d347d997/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.268551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65659929-becd-4212-a074-902f978de64a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.278677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 780/1589] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.280070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65659929-becd-4212-a074-902f978de64a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.286864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2e10a0973e431db6cc11cc924fd826, user_id=d1ac1cdd046a4ae4b9f41b75d347d997, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.289543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.289708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65659929-becd-4212-a074-902f978de64a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.294268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.294913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.295204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.295476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.296235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.297913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f5f6a-53ed-43ba-b5a1-e592b32d6139 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.303470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 810/1590] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:19 2022] PUT /identity/v3/projects/bd2e10a0973e431db6cc11cc924fd826/users/d1ac1cdd046a4ae4b9f41b75d347d997/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.306289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.308292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 781/1591] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.309387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3fa33-b8b6-4430-bd34-d58ad48b906a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.309676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3fa33-b8b6-4430-bd34-d58ad48b906a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.309974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3fa33-b8b6-4430-bd34-d58ad48b906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.327583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0fa3fa33-b8b6-4430-bd34-d58ad48b906a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:19.328816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0fa3fa33-b8b6-4430-bd34-d58ad48b906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:19.375114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751903f-8612-4f96-a99a-74df5f400a17 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.380486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.381165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.381484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.381765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.382445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.384350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.387347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 782/1592] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:19.388605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f554a00-90f5-4977-9a69-0ad63ba0bb71 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.388899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f554a00-90f5-4977-9a69-0ad63ba0bb71 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.389177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f554a00-90f5-4977-9a69-0ad63ba0bb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.409368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 811/1593] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:19.409993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.419713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f554a00-90f5-4977-9a69-0ad63ba0bb71 None None] MFA Rules not processed for user `3791cb0aef234ac994507815453f7b63`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:19.420932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f554a00-90f5-4977-9a69-0ad63ba0bb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:19.421267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d2153-7e2f-415f-a952-bfba117a6ec9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.426117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.426577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.426776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.427050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.427609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.428420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.430247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.441546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 812/1594] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.442350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.449658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.456083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9481b8-cc5e-4a6d-865b-4c974c32fac4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.461140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.461825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.462112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.462391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.463097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.464669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.475579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 813/1595] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.476375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.488227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d60cc-1df5-4b4f-8d16-50a72320939e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.493111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.493379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 783/1596] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:19.493792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.494075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.494349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.494809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.495064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.495157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.495371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.496652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.510597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 814/1597] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.510597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.516877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:19.518075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ff100-ee50-46d6-ba04-5195ec7fa83d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.518196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:19.522568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.523436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.523720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.523995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] PATH_INFO: `/v3/projects/bc07c427133343d7b3afd29191a3af85/users/b3aea02fafe44f0fb7a654e39296e338/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.525957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 784/1598] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:19.526506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.527741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:19.539867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc07c427133343d7b3afd29191a3af85, user_id=b3aea02fafe44f0fb7a654e39296e338, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.542562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.556499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 815/1599] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:19 2022] PUT /identity/v3/projects/bc07c427133343d7b3afd29191a3af85/users/b3aea02fafe44f0fb7a654e39296e338/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.557808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.602278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198885c4-3ac7-4f95-85b8-09cdf08ed3d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.620273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e6e7-cdf3-44ae-9cde-6510d370e4eb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.624938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.625666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.625954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.626233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.626955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.628555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.638498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 816/1600] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.642827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.652505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136781c1-cba9-4965-92a9-112eb8e187b5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.654751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.655392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.655672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.655946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.657661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.658290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.658568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.658842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.659547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.661135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.665721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.667169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.670884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 817/1601] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:19.671637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.680290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 785/1602] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:19.680678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700361-f2b0-46f5-a96c-8cc789588338 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.681296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.685620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.686488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.686769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.687126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] PATH_INFO: `/v3/projects/f0c7a92dfcc847e086b94b2875d109db/users/d0f22ba036b84cb194732c02570e8568/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.691185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055fe7-3d54-4572-ae6d-524c0f87bf43 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.696553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.697226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.697544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.697822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.698509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.700473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.704846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c7a92dfcc847e086b94b2875d109db, user_id=d0f22ba036b84cb194732c02570e8568, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.707955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.725593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 818/1603] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:19 2022] PUT /identity/v3/projects/f0c7a92dfcc847e086b94b2875d109db/users/d0f22ba036b84cb194732c02570e8568/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.726252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.733291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 786/1604] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:19.733291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.783004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.803375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ba505-5d28-4ba3-bd01-cdccd4f78961 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.806406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ea86c-bdfc-4e77-a8f2-8d048ee101c0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:19.807715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.808653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.808776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.808959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.831475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:19.885053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:19.886104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.886388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.886664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] PATH_INFO: `/v3/projects/7122215625b64963ab03b66f4c9d8a7f/users/3753498f22d5443ab0c01473c91a263c/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.894297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.896144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a79019-58e9-4b5d-9fd2-5a1729e9c9d0 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.905743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7122215625b64963ab03b66f4c9d8a7f, user_id=3753498f22d5443ab0c01473c91a263c, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:19.908147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 787/1605] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:19.909406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:19.909724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:19.909889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:19.910578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:19.926192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 819/1606] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:19 2022] PUT /identity/v3/projects/7122215625b64963ab03b66f4c9d8a7f/users/3753498f22d5443ab0c01473c91a263c/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:19.927495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:19.948724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] MFA Rules not processed for user `b3aea02fafe44f0fb7a654e39296e338`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:19.950507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:20.003020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093da3b1-f313-4d7d-9dae-6a20291e1660 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.008316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.009106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.009546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.009928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.010751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.012782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.023276 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 788/1607] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:20.024200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.029227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 820/1608] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:20.030230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.040952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4a5d7-641c-4424-b0f6-29415e72024d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.041742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214c498-00b8-4273-8d53-4a8f003741f7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.045973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.046549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.047051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.047340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.047501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.047706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.047811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.048085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] PATH_INFO: `/v3/projects/91b118a5d8ec4946a4da74e0403c8c20/users/32f38cb761704726ab5b6a2289603a23/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.048401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.051000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3198d-8fe8-447c-a877-84cf1d8fffb5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.070849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b118a5d8ec4946a4da74e0403c8c20, user_id=32f38cb761704726ab5b6a2289603a23, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.072480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:20.072480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:32:20.072480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:32:20.074654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d1461-7282-4d50-9c42-9f7421ba9707 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.078420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 821/1609] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:20.078420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.078420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.078420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.093416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 789/1610] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:20 2022] PUT /identity/v3/projects/91b118a5d8ec4946a4da74e0403c8c20/users/32f38cb761704726ab5b6a2289603a23/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:20.106159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.106531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.106719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.127010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] MFA Rules not processed for user `d0f22ba036b84cb194732c02570e8568`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:20.129052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:20.134962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:20.136461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:20.231981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:20.266761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 790/1611] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:20.267966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.289247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250e3e7-74d8-4329-a98b-267a7b2d721a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.293999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.294602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.294936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.295773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.296015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.298002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.303169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 822/1612] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:20.307572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.324487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c73186-4de4-4f6c-844d-a3a9754247c7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.330940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.332566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.333760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.334412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] PATH_INFO: `/v3/projects/4a17ad13530d4d25a0d3c1f583526bf6/users/e66082f9ca364548804389deb5cfa9df/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.339543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 791/1613] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:20.340605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.354409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:20.358812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a17ad13530d4d25a0d3c1f583526bf6, user_id=e66082f9ca364548804389deb5cfa9df, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.363414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6e3fa-95f3-4f7a-9f1f-158872fb3566 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.389132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 823/1614] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:20 2022] PUT /identity/v3/projects/4a17ad13530d4d25a0d3c1f583526bf6/users/e66082f9ca364548804389deb5cfa9df/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:20.393687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.394016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.394259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.450666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] MFA Rules not processed for user `32f38cb761704726ab5b6a2289603a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:20.454836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:20.478259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc0718-f416-4bfe-9ac2-9018718c44d5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.577067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.577833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.578131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.579165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.580516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.583098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.599226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 792/1615] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:20.599226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.607264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 824/1616] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:20.608514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.622362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa2d35-ba84-4094-aea4-0ec391ff34c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.637600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.638684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.639201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.639690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.640661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.643054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.685494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 825/1617] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/projects => generated 355 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:20.686751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.695883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857c21a-5821-4719-a466-57119628b08c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.735337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:20.772352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.773179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.773803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.774183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.824871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc5b-c451-4329-b460-9ef36594e8e9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.838373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.839078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.839352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.839631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.840332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.843282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5926-b200-4784-862c-19b48f8d1c12 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.879072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.880545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 826/1618] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:20.881939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.882075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.882238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.883172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.906334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 793/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:20.907044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:20.907277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.908599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:20.919481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 827/1620] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:20.920539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.923186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d697a3-a222-418d-803f-eb9236ebf4e5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:20.931421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:20.933210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:20.933568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:20.933854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] PATH_INFO: `/v3/projects/b698cb8f080e4e248d7999fe7abed746/users/052b9163703445a08330735f40d722c2/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:20.957501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b698cb8f080e4e248d7999fe7abed746, user_id=052b9163703445a08330735f40d722c2, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:20.961541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:20.983807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 794/1621] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:20 2022] PUT /identity/v3/projects/b698cb8f080e4e248d7999fe7abed746/users/052b9163703445a08330735f40d722c2/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:20.987858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:20.991836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:21.015497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589fb13-b22a-4f5b-84ff-f1aba41e629b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.097619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17b834-98d1-4661-8c53-8d972e95cfad admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.107747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.108420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.108703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.108979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.114610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.115172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.115457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.116767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.117246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.121680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.165292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 795/1622] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:20 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:21.169005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.190790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdad5c-b96f-4f87-94e8-321050a835cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.201892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.202577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.202867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.203145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.204409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.206075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.213973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.216778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.230140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 796/1624] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:21.230470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 828/1624] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:21.233464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.234630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.249749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51da3b-23f0-4119-ad4d-b460dbf7f6bb service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.259930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.260833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.261171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.261510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.262245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.264708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.287902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 829/1625] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:21.291804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.320336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d274a4b-4c63-467f-aa03-6286d91cd5bd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.326560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:21.354178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.355880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.356168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.356502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.357248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.363830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56204e16-3b11-454b-9b59-06246d535bf3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.364513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.372918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.373951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.373951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.374172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.397717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.399732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 830/1626] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:21.402068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.402296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.424813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 797/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:21.425505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.448424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006bf68-82c5-4467-992c-4dba7523b349 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.448854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231bcc8-a1a2-4810-ad3d-cef62e2c77bc service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.459515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.460423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.460708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.461003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] PATH_INFO: `/v3/projects/b698cb8f080e4e248d7999fe7abed746/users/052b9163703445a08330735f40d722c2/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.465867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b698cb8f080e4e248d7999fe7abed746, user_id=052b9163703445a08330735f40d722c2, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.466070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.467998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.468488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.468606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.469850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] PATH_INFO: `/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea/users/1578fb03d949465fbd1f6b6efc095ff9/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.505807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 798/1628] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:21 2022] PUT /identity/v3/projects/b698cb8f080e4e248d7999fe7abed746/users/052b9163703445a08330735f40d722c2/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:21.509855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.522444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d6e64f88c34fc8aa7b033c3b9b54ea, user_id=1578fb03d949465fbd1f6b6efc095ff9, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.525097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1177ddf-518f-4146-9759-539a3aa1dbf8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.555340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 831/1629] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:21 2022] PUT /identity/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea/users/1578fb03d949465fbd1f6b6efc095ff9/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:21.559259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.559808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.560238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.630999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] MFA Rules not processed for user `052b9163703445a08330735f40d722c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:21.633478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:21.653019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397846e-3926-4202-8b5e-51839bf3006f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.773587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:21.798928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.799842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.805637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.805959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] PATH_INFO: `/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb/users/e6e54193f298417faa2f187f945c65f1/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.831896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 832/1630] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:21.833433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.858180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cca85b7c2f41da8c7289f1c673c8cb, user_id=e6e54193f298417faa2f187f945c65f1, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.858966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dfbf1-a85d-4371-8d0a-79750300acbe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:21.861902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.870008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:21.870745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:21.873183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:21.873741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:21.874624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:21.876591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:21.892043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 799/1631] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:21 2022] PUT /identity/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb/users/e6e54193f298417faa2f187f945c65f1/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:21.892843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.909481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 833/1632] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:21.910400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:21.945789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fec222-c52f-481f-a429-62758a343897 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.017909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e999af-1610-40af-a823-fce5ab76902b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.024697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.025536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.026297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.026449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.051696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:22.084677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.085402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.085832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.086114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.086848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.088462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.120493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 834/1633] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:22.124965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:22.142460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e07f2d-53e7-4b61-84be-1a471a7b963b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.158612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.159535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.159820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.160102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] PATH_INFO: `/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea/users/1578fb03d949465fbd1f6b6efc095ff9/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.162860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d6e64f88c34fc8aa7b033c3b9b54ea, user_id=1578fb03d949465fbd1f6b6efc095ff9, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.167234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.235966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 835/1634] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:22 2022] PUT /identity/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea/users/1578fb03d949465fbd1f6b6efc095ff9/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:22.235966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:22.238125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.238125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924d393-0398-4990-8f85-af12dff7f54f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.302485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 800/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:22.304000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b03b-55ad-4933-b5bf-e2353d4c687b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.304291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b03b-55ad-4933-b5bf-e2353d4c687b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.304584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b03b-55ad-4933-b5bf-e2353d4c687b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.418834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3863b0-18df-4a43-ac9b-a3a9e4eb6c88 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.434277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e9f6b03b-55ad-4933-b5bf-e2353d4c687b None None] MFA Rules not processed for user `1578fb03d949465fbd1f6b6efc095ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:22.435286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.435518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e9f6b03b-55ad-4933-b5bf-e2353d4c687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:22.436520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.436749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.437183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] PATH_INFO: `/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb/users/e6e54193f298417faa2f187f945c65f1/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.441086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cca85b7c2f41da8c7289f1c673c8cb, user_id=e6e54193f298417faa2f187f945c65f1, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.453317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.491589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 836/1636] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:22 2022] PUT /identity/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb/users/e6e54193f298417faa2f187f945c65f1/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:22.491589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:22.605705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500e82c-83a8-4e1a-bdc0-c78d53f072a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.611361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.612037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.612317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.615970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.642828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:22.665745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 801/1637] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:22.667474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.667880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.669428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.737632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] MFA Rules not processed for user `e6e54193f298417faa2f187f945c65f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:22.740293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:22.752045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.758268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.800502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 837/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:22.801558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:22.827405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7f632-c841-41c8-80c2-f65a3091039d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:22.838157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:22.838823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:22.838823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:22.839118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:22.864394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 802/1639] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:22.964782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:22.966613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:22.982632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 838/1640] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:23.068211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:23.082284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642877bd-0cd7-4996-8e4c-9bee2720077c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:23.090246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:23.091239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:23.091637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:23.092028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:23.119011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:23.121925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:23.148012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 803/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:23 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:23.515039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:23.547670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08627-4cef-4c49-b7db-7cc04c179cca service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:23.563478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:23.564454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:23.564780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:23.566796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:23.866239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:23.872336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:23.906724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 839/1642] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:23 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:24.339188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:24.345801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:24.367439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6331f-6012-4007-9ae0-be730ca96421 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:24.394669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1578fb03d949465fbd1f6b6efc095ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:24.395366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:24.395663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:24.395954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:24.396693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:24.402772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:24.405111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:24.479882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 804/1643] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:24.483514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:24.508629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5aeb5-d9fb-4232-badd-456d87573c8f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:24.529219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1578fb03d949465fbd1f6b6efc095ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:24.529936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:24.530228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:24.530513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:24.532155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:24.534369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cc24f-2cf8-441e-a863-e94112fc2ff5 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:24.632227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 840/1644] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:24 2022] POST /identity/v3/projects => generated 353 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:24.633596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:24.651902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534a298-492f-4d99-bcbe-2bd234a5a676 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:24.829962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1578fb03d949465fbd1f6b6efc095ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:24.830685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:24.830978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:24.831381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:24.832132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:24.832449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:24.834450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f701d-08c9-4359-a0a2-50de7427efc7 tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:24.882428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 805/1645] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:25.059510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872d9e-71f0-4fe7-aa44-c740582001d4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.059964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872d9e-71f0-4fe7-aa44-c740582001d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.060235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f872d9e-71f0-4fe7-aa44-c740582001d4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.061828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 841/1646] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 14:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:25.080715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863149d-2a06-4f65-a9ed-ba88ed38daca None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.081028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863149d-2a06-4f65-a9ed-ba88ed38daca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.081288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863149d-2a06-4f65-a9ed-ba88ed38daca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.197198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3863149d-2a06-4f65-a9ed-ba88ed38daca None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:25.204686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3863149d-2a06-4f65-a9ed-ba88ed38daca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:25.281254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f872d9e-71f0-4fe7-aa44-c740582001d4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:25.305949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f872d9e-71f0-4fe7-aa44-c740582001d4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:25.322955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1578fb03d949465fbd1f6b6efc095ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:25.325605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.332410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.332891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] PATH_INFO: `/v3/projects/b51f1c0ccd564868b7e657a5f7dfe31c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.335937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b51f1c0ccd564868b7e657a5f7dfe31c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:25.340291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:25.340291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: warnings.warn(msg) Jul 12 14:32:25.340291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: Jul 12 14:32:25.349400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:25.445927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 806/1647] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 14:32:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:25.448025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011cd-d52a-43b8-897d-a91a5a2ec988 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.448610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011cd-d52a-43b8-897d-a91a5a2ec988 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.449184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e011cd-d52a-43b8-897d-a91a5a2ec988 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.450347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:25.456741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 807/1648] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 14:32:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:25.494891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.494891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.494891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.494891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:25.496218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:25.523870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 808/1649] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:25.537381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:25.635505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51f1c0ccd564868b7e657a5f7dfe31c', 'request_id': 'req-8d7ea28b-8c60-4809-aa0d-61272b5155ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:25.651118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51f1c0ccd564868b7e657a5f7dfe31c', 'request_id': 'req-8d7ea28b-8c60-4809-aa0d-61272b5155ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:25.668491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51f1c0ccd564868b7e657a5f7dfe31c', 'request_id': 'req-8d7ea28b-8c60-4809-aa0d-61272b5155ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:25.716116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] The token cache is being invalidate because project b51f1c0ccd564868b7e657a5f7dfe31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:25.718156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:25.719233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8d7ea28b-8c60-4809-aa0d-61272b5155ca tempest-VolumeQuotaClassesTest-1402499300 tempest-VolumeQuotaClassesTest-1402499300-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:25.725490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 842/1650] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:25 2022] DELETE /identity/v3/projects/b51f1c0ccd564868b7e657a5f7dfe31c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:25.730015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139ba0c-8706-4763-acd7-5591c08573b3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:25.751587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:25.866922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:25.867610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:25.867894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:25.868172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:25.875804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:26.096407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:26.103329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:26.135198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 809/1651] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:26.335352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:26.336230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:26.336731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:26.419333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] MFA Rules not processed for user `e66082f9ca364548804389deb5cfa9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:26.423615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:26.479969 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:26.481547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:26.666043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9409bd-85ba-4154-96f5-e4fa2cd8b21b service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:26.673446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:26.674574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:26.675022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:26.675697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] PATH_INFO: `/v3/users/b3aea02fafe44f0fb7a654e39296e338` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:26.679144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3aea02fafe44f0fb7a654e39296e338)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:26.680459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 14:32:26.680459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jul 12 14:32:26.680459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: Jul 12 14:32:26.681347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 843/1652] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:26.683429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:26.684670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:26.686206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:26.725769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7a568-0fdb-4165-a34f-e577c28d4a2a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:26.736854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:26.741425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:26.741676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:26.741978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:26.742767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:26.746574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:26.824188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 844/1653] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:26 2022] POST /identity/v3/projects => generated 403 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:26.826237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:26.842722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3aea02fafe44f0fb7a654e39296e338'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:26.863144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3aea02fafe44f0fb7a654e39296e338'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:26.865722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3aea02fafe44f0fb7a654e39296e338'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:26.867380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:26.919154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3aea02fafe44f0fb7a654e39296e338'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:26.938918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8c5cb9ff-aa22-4312-8228-9b3dfc6d2f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3aea02fafe44f0fb7a654e39296e338'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:26.949988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 810/1654] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:26 2022] DELETE /identity/v3/users/b3aea02fafe44f0fb7a654e39296e338 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:26.952164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:26.952595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:26.952937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.005629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:27.007134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:27.042088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24c0fc-adf2-46bb-a3a1-1bb3e6cd4455 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.107762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:27.112229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 811/1655] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:27.197231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:27.198760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:27.239294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.242324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.242324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.242324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.242324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.243097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:27.326394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 845/1656] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:26 2022] POST /identity/v3/users => generated 308 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:27.328041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:27.338460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:27.398604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd935ab8-87b0-4e44-a748-9bdaca65273f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.407103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.409460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.410338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.411460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.416062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.418534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:27.460129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9202b08-5bee-4f75-82cb-18d33eaa09ab None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.474291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 846/1657] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:27.475076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.475342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:27.476208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.485902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.487008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.495646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ca3cf-9d02-4e0b-b019-efd294e78dc4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.511780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.514510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.515045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.515374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] PATH_INFO: `/v3/projects/dcee4944f0604ea094e44eb774021695/users/6897ee6ffcf548bab08a79416aa4c18e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.520625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.522408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:27.564931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 812/1658] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:32:27.590632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcee4944f0604ea094e44eb774021695, user_id=6897ee6ffcf548bab08a79416aa4c18e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.595254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:27.649990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 847/1659] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:27 2022] PUT /identity/v3/projects/dcee4944f0604ea094e44eb774021695/users/6897ee6ffcf548bab08a79416aa4c18e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:27.654060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:27.668712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928587d-e3f1-4f5b-8e10-3d3684f0908e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.871331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.872189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.872577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.872975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.873861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.876219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964a43-2cc8-4c3e-a63a-45379c68c527 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:27.910237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 813/1660] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:27.911651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:27.932797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a811c08-1492-440d-8edf-da6b8e2dd7b1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:27.964939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:27.966084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:27.966528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:27.966971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] PATH_INFO: `/v3/projects/dcee4944f0604ea094e44eb774021695/users/6897ee6ffcf548bab08a79416aa4c18e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:27.972661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcee4944f0604ea094e44eb774021695, user_id=6897ee6ffcf548bab08a79416aa4c18e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:27.977946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:28.002884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 848/1661] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:27 2022] PUT /identity/v3/projects/dcee4944f0604ea094e44eb774021695/users/6897ee6ffcf548bab08a79416aa4c18e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:28.268190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890cc7d-7b2a-4bbe-bbe8-4e3df192402d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:28.268823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890cc7d-7b2a-4bbe-bbe8-4e3df192402d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:28.270167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890cc7d-7b2a-4bbe-bbe8-4e3df192402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:28.315263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:28.340026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1890cc7d-7b2a-4bbe-bbe8-4e3df192402d None None] MFA Rules not processed for user `b22129ca8c714db6ad44dee94e6099e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:28.342197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1890cc7d-7b2a-4bbe-bbe8-4e3df192402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:28.480499 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:28.482712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:28.499674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 814/1662] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:28.555700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c815e9-7918-411d-beff-73ea954cfc51 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:28.556225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c815e9-7918-411d-beff-73ea954cfc51 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:28.556689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c815e9-7918-411d-beff-73ea954cfc51 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:28.558750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 815/1663] 10.222.0.25 () {54 vars in 961 bytes} [Tue Jul 12 14:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:32:28.589029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:28.589317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:28.589921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:28.653807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:28.655084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:28.719940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b900ab-c7a8-4413-afc4-a0725e934f40 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:28.743567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:28.746589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:28.746897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:28.750032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] PATH_INFO: `/v3/users/3791cb0aef234ac994507815453f7b63` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:28.755063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3791cb0aef234ac994507815453f7b63)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:28.762213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:28.914441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3791cb0aef234ac994507815453f7b63'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:28.922751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 816/1664] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Jul 12 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:28.925280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3791cb0aef234ac994507815453f7b63'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:28.937078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:28.937186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:28.962731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3791cb0aef234ac994507815453f7b63'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:28.966395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3791cb0aef234ac994507815453f7b63'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:28.969233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e593a5-a1d9-45fe-bd49-cc243fe0cc54 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:28.992972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:28.994177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:28.994813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:28.995595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:28.995710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28be0049-5e0d-4499-a046-c8391d03b732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3791cb0aef234ac994507815453f7b63'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:28.999059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:29.017757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 849/1665] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:28 2022] DELETE /identity/v3/users/3791cb0aef234ac994507815453f7b63 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:29.019227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077873b-4bce-42b1-b03e-fea0a5512575 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:29.019471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077873b-4bce-42b1-b03e-fea0a5512575 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:29.019914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077873b-4bce-42b1-b03e-fea0a5512575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:29.039936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d077873b-4bce-42b1-b03e-fea0a5512575 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:29.041449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d077873b-4bce-42b1-b03e-fea0a5512575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:29.050498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:29.061729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:29.117437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 817/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:29.195996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 850/1667] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:29.229378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:29.230842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:29.451498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493b5e-0a88-47be-b60a-8c8093258a90 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:29.537840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:29.564284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:29.564945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:29.565233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:29.566364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:29.576756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:29.579061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:29.593258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 818/1668] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:29.991633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:29.992237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:29.992556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:30.063343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] MFA Rules not processed for user `6897ee6ffcf548bab08a79416aa4c18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:30.064975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:30.166740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 851/1669] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:30.523023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:30.526494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:30.555379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8076-59f5-4ef8-b006-05039a6d872e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:30.570467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:30.572538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:30.572905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:30.573130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] PATH_INFO: `/v3/projects/bc07c427133343d7b3afd29191a3af85` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:30.577103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc07c427133343d7b3afd29191a3af85)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:30.581698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:30.757845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc07c427133343d7b3afd29191a3af85', 'request_id': 'req-2cb941aa-7421-4050-b63a-7b645bc0e112'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:30.799132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc07c427133343d7b3afd29191a3af85', 'request_id': 'req-2cb941aa-7421-4050-b63a-7b645bc0e112'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:30.818256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc07c427133343d7b3afd29191a3af85', 'request_id': 'req-2cb941aa-7421-4050-b63a-7b645bc0e112'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:30.866346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] The token cache is being invalidate because project bc07c427133343d7b3afd29191a3af85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:30.866653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:30.869588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cb941aa-7421-4050-b63a-7b645bc0e112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:30.875502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 819/1670] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:30 2022] DELETE /identity/v3/projects/bc07c427133343d7b3afd29191a3af85 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:30.879100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:30.895616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d329d-2712-4d0b-867b-672fe2c07697 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:30.910799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293612b-94e1-45a8-b487-91e1d24acad6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:30.911089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293612b-94e1-45a8-b487-91e1d24acad6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:30.911468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293612b-94e1-45a8-b487-91e1d24acad6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:30.915452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 820/1671] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 14:32:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:30.929475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:30.929475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:30.929475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:30.971199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:30.980664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:31.087025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:31.087923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:31.088204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:31.088479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] PATH_INFO: `/v3/users/1578fb03d949465fbd1f6b6efc095ff9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:31.090458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1578fb03d949465fbd1f6b6efc095ff9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:31.092376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:31.126618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:31.209242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 821/1672] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:31.221714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1578fb03d949465fbd1f6b6efc095ff9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:31.241218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1578fb03d949465fbd1f6b6efc095ff9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:31.259989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1578fb03d949465fbd1f6b6efc095ff9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:31.264269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1578fb03d949465fbd1f6b6efc095ff9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:31.265296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:31.266801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:31.307089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-07722f97-fa43-45c5-9cf0-f6e0544039b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1578fb03d949465fbd1f6b6efc095ff9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:31.338110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 852/1673] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:30 2022] DELETE /identity/v3/users/1578fb03d949465fbd1f6b6efc095ff9 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:31.359451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb6036-16c3-425e-b0ef-21fb9dcdb96d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:31.370471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:31.519652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:31.526177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:31.526177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:31.526177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:31.530993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:31.731654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:31.741776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ebaac-e26a-45e3-aef8-2809239db726 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:31.765022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 822/1674] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:32:32.157917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821760-8635-4a5b-b2e0-cceaefc4a6d7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.158409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821760-8635-4a5b-b2e0-cceaefc4a6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.158481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821760-8635-4a5b-b2e0-cceaefc4a6d7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.160256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 853/1675] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:32.193140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf640ac-ceb2-4ae4-87b3-9edabf38a161 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.193746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf640ac-ceb2-4ae4-87b3-9edabf38a161 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.194137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf640ac-ceb2-4ae4-87b3-9edabf38a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.225642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ecf640ac-ceb2-4ae4-87b3-9edabf38a161 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:32.227023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ecf640ac-ceb2-4ae4-87b3-9edabf38a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:32.407479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 823/1676] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:32.473463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2059-e5dd-4ab8-be6b-f1bfafa9f5c5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.474119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2059-e5dd-4ab8-be6b-f1bfafa9f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.474778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2059-e5dd-4ab8-be6b-f1bfafa9f5c5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.476394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 854/1677] 10.222.0.25 () {54 vars in 960 bytes} [Tue Jul 12 14:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:32.482503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.483242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.483723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.526960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:32.528630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:32.553494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 824/1678] 10.222.0.25 () {58 vars in 1057 bytes} [Tue Jul 12 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:32.563125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2059-e5dd-4ab8-be6b-f1bfafa9f5c5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:32.587649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2059-e5dd-4ab8-be6b-f1bfafa9f5c5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:32.600162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:32.601366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:32.601863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:32.602343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:32.622655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:32.624600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:32.661399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 855/1679] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:33.054170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:33.056166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:33.175716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:33.226184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744125-f5da-40e9-8cdf-fb39c7dd3b54 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:33.243483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:33.244615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:33.245100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:33.245623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] PATH_INFO: `/v3/projects/2db99ccb0a8845a48cba15c209b8a425` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:33.247318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db99ccb0a8845a48cba15c209b8a425)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:33.262159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:33.377165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db99ccb0a8845a48cba15c209b8a425', 'request_id': 'req-5a8b71d2-432a-47cd-af82-9c2947690aa4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.398499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db99ccb0a8845a48cba15c209b8a425', 'request_id': 'req-5a8b71d2-432a-47cd-af82-9c2947690aa4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.417912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db99ccb0a8845a48cba15c209b8a425', 'request_id': 'req-5a8b71d2-432a-47cd-af82-9c2947690aa4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.467622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] The token cache is being invalidate because project 2db99ccb0a8845a48cba15c209b8a425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:33.470317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.472224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a8b71d2-432a-47cd-af82-9c2947690aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.497709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 825/1680] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:33 2022] DELETE /identity/v3/projects/2db99ccb0a8845a48cba15c209b8a425 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:33.502218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:33.534161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bfd91-cfa8-4953-8736-89173974f4c4 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:33.777665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:33.779189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:33.779669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:33.780109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] PATH_INFO: `/v3/users/052b9163703445a08330735f40d722c2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:33.782772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=052b9163703445a08330735f40d722c2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:33.784752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:33.935875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052b9163703445a08330735f40d722c2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:33.976709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052b9163703445a08330735f40d722c2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:34.008776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052b9163703445a08330735f40d722c2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:34.009922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052b9163703445a08330735f40d722c2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:34.018835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:34.028798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bed99f25-26a5-4f59-82d6-942d723b1f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052b9163703445a08330735f40d722c2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:34.049753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 856/1681] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:33 2022] DELETE /identity/v3/users/052b9163703445a08330735f40d722c2 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:34.482139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:34.483359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:34.488007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:34.543842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] MFA Rules not processed for user `d1ac1cdd046a4ae4b9f41b75d347d997`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:34.552319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:34.693487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 826/1682] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:34.705506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:34.707754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:34.708211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:34.761843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] MFA Rules not processed for user `d1ac1cdd046a4ae4b9f41b75d347d997`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:34.763572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:34.779683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 857/1683] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:34.786504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:34.788040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:34.943523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:34.995604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293c24f-2d09-42bf-b041-63ba1fa468c5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:35.012046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:35.012735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.013026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.015856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.039108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:35.046863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:35.079077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 827/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:35.271186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:35.272981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.440302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9ffa0-d20a-4622-9538-af33c4c2200e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:35.451030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:35.454053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.454503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.454793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] PATH_INFO: `/v3/projects/b698cb8f080e4e248d7999fe7abed746` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.460903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b698cb8f080e4e248d7999fe7abed746)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:35.465413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:35.541431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:35.542913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.572024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63535c8-0fc7-43f2-8f85-e33e76af6676 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:35.575434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b698cb8f080e4e248d7999fe7abed746', 'request_id': 'req-8a024fd8-078a-461b-a94b-d9e29fd287dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.587329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b698cb8f080e4e248d7999fe7abed746', 'request_id': 'req-8a024fd8-078a-461b-a94b-d9e29fd287dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.595422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b698cb8f080e4e248d7999fe7abed746', 'request_id': 'req-8a024fd8-078a-461b-a94b-d9e29fd287dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.623630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] The token cache is being invalidate because project b698cb8f080e4e248d7999fe7abed746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:35.624013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.625155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8a024fd8-078a-461b-a94b-d9e29fd287dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.628144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 858/1685] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:35 2022] DELETE /identity/v3/projects/b698cb8f080e4e248d7999fe7abed746 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:35.630042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1f99-d9ed-48da-a867-ca72adaf595f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.630428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1f99-d9ed-48da-a867-ca72adaf595f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.631839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f1f99-d9ed-48da-a867-ca72adaf595f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.663241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2c4f1f99-d9ed-48da-a867-ca72adaf595f None None] MFA Rules not processed for user `b22129ca8c714db6ad44dee94e6099e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:35.664752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2c4f1f99-d9ed-48da-a867-ca72adaf595f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.715794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:35.716766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.717977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.718479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] PATH_INFO: `/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.723952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d6e64f88c34fc8aa7b033c3b9b54ea)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:35.729541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:35.774105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 859/1686] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:35.779464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5598-2f51-4351-adfb-7c2b37052f16 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.779464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5598-2f51-4351-adfb-7c2b37052f16 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.779464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c5598-2f51-4351-adfb-7c2b37052f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:35.839386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a53c5598-2f51-4351-adfb-7c2b37052f16 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:35.841727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'request_id': 'req-f271d667-e0aa-446c-83b0-842fc5865582'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.842255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a53c5598-2f51-4351-adfb-7c2b37052f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:35.867983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'request_id': 'req-f271d667-e0aa-446c-83b0-842fc5865582'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.889113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d6e64f88c34fc8aa7b033c3b9b54ea', 'request_id': 'req-f271d667-e0aa-446c-83b0-842fc5865582'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.896648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:35.924356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] The token cache is being invalidate because project 67d6e64f88c34fc8aa7b033c3b9b54ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:35.924673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.927895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f271d667-e0aa-446c-83b0-842fc5865582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:35.934601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 828/1687] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:35 2022] DELETE /identity/v3/projects/67d6e64f88c34fc8aa7b033c3b9b54ea => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:35.956431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592df6d-0291-46d2-ad40-c6f3ebf71201 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:35.956723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592df6d-0291-46d2-ad40-c6f3ebf71201 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:35.957002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592df6d-0291-46d2-ad40-c6f3ebf71201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.009877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5592df6d-0291-46d2-ad40-c6f3ebf71201 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:36.027172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5592df6d-0291-46d2-ad40-c6f3ebf71201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.042945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 860/1688] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:36.053119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.053681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.054065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.055022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 829/1689] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:36.065310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.067991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.068652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.113150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:36.117001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.138260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 861/1690] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:36.142601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.152382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:36.154052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.164279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fc68b-db57-495a-b2be-1ed1f8dc8d59 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.185832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.186176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.186467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.186744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.194137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:36.195541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.197491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 830/1691] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:36.199672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.202393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.219355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.246244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920ffce-026b-4370-b3e6-fd81b1180355 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.259241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.262142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.262379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.262760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] PATH_INFO: `/v3/users/e66082f9ca364548804389deb5cfa9df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.265811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e66082f9ca364548804389deb5cfa9df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.265977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 862/1692] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:36.266901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.267124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.288219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe61d6-93be-4757-9087-f762ce741d83 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.301239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.302508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.304352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.304672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.308529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:36.308977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.313956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.337572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 863/1693] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:36.338565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.356730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a42b5-8be2-4a2b-a8d4-390bc518c9fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.386134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.386971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.387985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.389047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.389909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.392424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.400041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e66082f9ca364548804389deb5cfa9df'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:36.419971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e66082f9ca364548804389deb5cfa9df'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:36.420943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e66082f9ca364548804389deb5cfa9df'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:36.424108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:36.449496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e66082f9ca364548804389deb5cfa9df'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:36.451841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 864/1694] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:36.457234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.462314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.494656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e66082f9ca364548804389deb5cfa9df'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:36.515194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 831/1695] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:36 2022] DELETE /identity/v3/users/e66082f9ca364548804389deb5cfa9df => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:36.525474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.529602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:36.689649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d22c1e-ec1d-473e-9d97-8882601ec54a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.774434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f7ee1-d2d3-4acb-91fc-56e212a7a1bf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.782667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.783391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.783675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.783956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.784710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.791857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae0e3f-71cd-4ec1-a0cb-b89a21c0aac3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.845964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 832/1696] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/users => generated 292 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:36.847357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a093a1-a17c-426b-a670-11fa8d915d60 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.847649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a093a1-a17c-426b-a670-11fa8d915d60 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.847929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a093a1-a17c-426b-a670-11fa8d915d60 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.858544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 833/1697] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 14:32:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:36.862267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.862969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.863256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.863540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.865037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a093a1-a17c-426b-a670-11fa8d915d60 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.866091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.871009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64e6a9-e381-4d70-9a49-0da9b2190b4f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.896626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a093a1-a17c-426b-a670-11fa8d915d60 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:36.908769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:36.910092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.910438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.910712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.912816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:36.913306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:36.919314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:36.932231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 865/1698] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/projects => generated 391 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:36.939872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24d028-c267-474b-965c-99285c1d488e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:36.939872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24d028-c267-474b-965c-99285c1d488e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:36.940160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24d028-c267-474b-965c-99285c1d488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:36.954278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 834/1699] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:36.956055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:36.988892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f6b5-970d-4ca3-aca7-872f4b295de9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.004723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8d24d028-c267-474b-965c-99285c1d488e None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:37.010568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d24d028-c267-474b-965c-99285c1d488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.192447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 866/1700] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:37.193327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24d028-c267-474b-965c-99285c1d488e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.263268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.264062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.264359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.264641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.266294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.268256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.327738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 835/1701] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:36 2022] POST /identity/v3/users => generated 296 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:37.331722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.366167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91a8a-2266-4740-bd5b-134d894abc17 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.382835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.383565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.383835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.384099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.384852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.392330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.418586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 836/1702] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:32:37.418586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.422577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.428316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24d028-c267-474b-965c-99285c1d488e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.466156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.470364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.470665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.471886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.471886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.473366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.490449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb47bca-144c-42de-a750-3aba207975a8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.504867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.506289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.506289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.506499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.513077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 867/1703] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:37.513077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.529376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.538352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.540192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.571421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 837/1704] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:37.576681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.581061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d917-f1a7-4ffa-bd8e-a529bef09db0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.584967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.597953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.604872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.605441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.606330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] PATH_INFO: `/v3/projects/4a17ad13530d4d25a0d3c1f583526bf6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.611444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a17ad13530d4d25a0d3c1f583526bf6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.616855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.633427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77491e41-39d7-428d-ac53-ec3ec6c8a60b service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:37.649793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:37.650688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.650977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.651260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] PATH_INFO: `/v3/users/32f38cb761704726ab5b6a2289603a23` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.655124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f38cb761704726ab5b6a2289603a23)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:37.657108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:37.687293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:37.737086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a17ad13530d4d25a0d3c1f583526bf6', 'request_id': 'req-ed03965e-1497-4b12-b997-7e62da95bca4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.750698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a17ad13530d4d25a0d3c1f583526bf6', 'request_id': 'req-ed03965e-1497-4b12-b997-7e62da95bca4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.766817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a17ad13530d4d25a0d3c1f583526bf6', 'request_id': 'req-ed03965e-1497-4b12-b997-7e62da95bca4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.789754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f38cb761704726ab5b6a2289603a23'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.808888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] The token cache is being invalidate because project 4a17ad13530d4d25a0d3c1f583526bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:37.809191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.809805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f38cb761704726ab5b6a2289603a23'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.810549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f38cb761704726ab5b6a2289603a23'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.811452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.814245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 868/1705] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:37 2022] DELETE /identity/v3/projects/4a17ad13530d4d25a0d3c1f583526bf6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:37.818622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.823718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.846840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f38cb761704726ab5b6a2289603a23'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.860456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ccc1b0d3-9832-45d4-bb1d-e8bd8b35dc27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f38cb761704726ab5b6a2289603a23'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:37.871723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 838/1706] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:37 2022] DELETE /identity/v3/users/32f38cb761704726ab5b6a2289603a23 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:37.873481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:37.873776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:37.874467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:37.876348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 839/1707] 10.222.0.25 () {54 vars in 912 bytes} [Tue Jul 12 14:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:37.877482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:37.879580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:37.904857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:38.038481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03965e-1497-4b12-b997-7e62da95bca4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.038709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3deb60-7439-4d73-91bf-1b560d17586f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.044415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.045319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.045628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.045999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] PATH_INFO: `/v3/projects/edf4b030a96b4180bcd628ebe6aeb966/users/e911c0cc15694e7fad8bc4f5a99d1fd1/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.053411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.053955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.054242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.054516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.055250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:38.056860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:38.093248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 869/1708] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:38.093248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:38.093739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf4b030a96b4180bcd628ebe6aeb966, user_id=e911c0cc15694e7fad8bc4f5a99d1fd1, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:38.098174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.102035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d43b6-f4bf-4539-8769-59455989bc0b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:38.136046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f6f7-d017-444c-921a-2bed9221982d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.138107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 840/1709] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:37 2022] PUT /identity/v3/projects/edf4b030a96b4180bcd628ebe6aeb966/users/e911c0cc15694e7fad8bc4f5a99d1fd1/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:38.140143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55d818-6feb-44a1-9c26-f51cb3b1ce29 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.140522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55d818-6feb-44a1-9c26-f51cb3b1ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.140961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55d818-6feb-44a1-9c26-f51cb3b1ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.161967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8d55d818-6feb-44a1-9c26-f51cb3b1ce29 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:38.163747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8d55d818-6feb-44a1-9c26-f51cb3b1ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.248356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.249302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.249715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.250517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] PATH_INFO: `/v3/users/6897ee6ffcf548bab08a79416aa4c18e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.251267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6897ee6ffcf548bab08a79416aa4c18e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:38.253207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:38.259231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 841/1710] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:38.272186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a63ecf-b4b0-4eab-b1e5-f7197aef38e2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.272748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a63ecf-b4b0-4eab-b1e5-f7197aef38e2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.273181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a63ecf-b4b0-4eab-b1e5-f7197aef38e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.302132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-02a63ecf-b4b0-4eab-b1e5-f7197aef38e2 None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:38.303386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-02a63ecf-b4b0-4eab-b1e5-f7197aef38e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.320709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:38.356246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6897ee6ffcf548bab08a79416aa4c18e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:38.371772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6897ee6ffcf548bab08a79416aa4c18e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:38.391928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6897ee6ffcf548bab08a79416aa4c18e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:38.393448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6897ee6ffcf548bab08a79416aa4c18e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:38.403701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6897ee6ffcf548bab08a79416aa4c18e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:38.413134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 870/1711] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:38 2022] DELETE /identity/v3/users/6897ee6ffcf548bab08a79416aa4c18e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:38.414113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:38.415512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.427461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 842/1712] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:38.428849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8416aa9-4597-41ef-88d8-645168ae9723 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.429577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8416aa9-4597-41ef-88d8-645168ae9723 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.429577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8416aa9-4597-41ef-88d8-645168ae9723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.468768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a8416aa9-4597-41ef-88d8-645168ae9723 None None] MFA Rules not processed for user `e911c0cc15694e7fad8bc4f5a99d1fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:38.470710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a8416aa9-4597-41ef-88d8-645168ae9723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.489020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:38.591956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0aa4b2-fdc5-44f9-94fb-e7c1a00bebc5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.597729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.598666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.598975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.599237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] PATH_INFO: `/v3/projects/04a8ce3db02b40a9abaaf6d69c6a2309/users/ac00fffa0ac04728a442234dc8736f15/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.653087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a8ce3db02b40a9abaaf6d69c6a2309, user_id=ac00fffa0ac04728a442234dc8736f15, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:38.656968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:38.681234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 843/1713] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:38.681437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 871/1714] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:38 2022] PUT /identity/v3/projects/04a8ce3db02b40a9abaaf6d69c6a2309/users/ac00fffa0ac04728a442234dc8736f15/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:38.681437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:38.681583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f661f-a904-4966-b2f8-762000bad512 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.681583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f661f-a904-4966-b2f8-762000bad512 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.681583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f661f-a904-4966-b2f8-762000bad512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.681751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.738180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-063f661f-a904-4966-b2f8-762000bad512 None None] MFA Rules not processed for user `3753498f22d5443ab0c01473c91a263c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:38.739417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-063f661f-a904-4966-b2f8-762000bad512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.790089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce57f3c-db28-4172-9808-a05a820a5a2d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.808591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.811679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.813353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.813867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.826457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:38.923621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 844/1715] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:38.930982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f661f-a904-4966-b2f8-762000bad512 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:38.955295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f661f-a904-4966-b2f8-762000bad512 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:38.961143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:38.962833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:38.963658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:38.964065 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:38.987518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:38.992875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.027310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.031867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.035653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 845/1716] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:39.043043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.045848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 872/1717] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:39.046068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.046795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.048144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.069320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e327-e649-4df8-930b-62063e3742ed service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.088335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:39.089026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.089313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.089642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.090643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.092628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d5a74-9f57-496c-94c9-b12c1ed3565d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.112250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:39.148103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 873/1718] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:39 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:39.152350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.153261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.153660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.200832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] MFA Rules not processed for user `ac00fffa0ac04728a442234dc8736f15`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:39.202385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.240248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab925ab-0524-4821-8813-ab6f62d01b15 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.274790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:39.390792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 874/1719] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:39.392462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.395166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.425723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:39.426399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.426726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.426991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.434740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.476523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.478417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.500788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 846/1720] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:39.501574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.505965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:39.604388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f756e47-1350-4552-ad69-fad158c9b23e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.618597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:39.619286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.619702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.619847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.657770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57b87f-1217-4365-b45d-ee955b982eb7 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.676222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:39.677122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:39.677437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:39.677723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] PATH_INFO: `/v3/projects/dcee4944f0604ea094e44eb774021695` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:39.679089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcee4944f0604ea094e44eb774021695)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.681098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.768251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:39.769933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:39.801036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 875/1721] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:39.801789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.816450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcee4944f0604ea094e44eb774021695', 'request_id': 'req-b16107ac-0d44-43c8-9441-fea75c6cdc3e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:39.827945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adc58f-7442-4780-bd34-ebc0ea294656 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:39.831401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcee4944f0604ea094e44eb774021695', 'request_id': 'req-b16107ac-0d44-43c8-9441-fea75c6cdc3e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:39.840412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:39.843792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcee4944f0604ea094e44eb774021695', 'request_id': 'req-b16107ac-0d44-43c8-9441-fea75c6cdc3e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:39.870418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] The token cache is being invalidate because project dcee4944f0604ea094e44eb774021695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:39.870711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:39.872487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:39.882086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 847/1722] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:39 2022] DELETE /identity/v3/projects/dcee4944f0604ea094e44eb774021695 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:39.882086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:39.885654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.015013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.020420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.020766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.021214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.022090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.026101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.076537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:40.100125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 876/1723] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:39 2022] POST /identity/v3/users => generated 299 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:40.100863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.102790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.128852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeac115-bf53-47fb-9bbe-1d42c794de75 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.151235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.152142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.152427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.152706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] PATH_INFO: `/v3/projects/91b118a5d8ec4946a4da74e0403c8c20` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.158089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b118a5d8ec4946a4da74e0403c8c20)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.162938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.177490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16107ac-0d44-43c8-9441-fea75c6cdc3e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.203214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.205694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.208773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.209701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.210606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.219020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.326242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b118a5d8ec4946a4da74e0403c8c20', 'request_id': 'req-50496dc5-e8f1-495f-b5ab-4a431d8649bc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:40.337831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b118a5d8ec4946a4da74e0403c8c20', 'request_id': 'req-50496dc5-e8f1-495f-b5ab-4a431d8649bc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:40.347464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b118a5d8ec4946a4da74e0403c8c20', 'request_id': 'req-50496dc5-e8f1-495f-b5ab-4a431d8649bc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:40.372467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] The token cache is being invalidate because project 91b118a5d8ec4946a4da74e0403c8c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:40.373098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:40.374065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-50496dc5-e8f1-495f-b5ab-4a431d8649bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:40.380329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 877/1724] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:40 2022] DELETE /identity/v3/projects/91b118a5d8ec4946a4da74e0403c8c20 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:40.382344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c330e-1df1-4406-8c8e-e0a3e98093f2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.382854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c330e-1df1-4406-8c8e-e0a3e98093f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.383352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c330e-1df1-4406-8c8e-e0a3e98093f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.391527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 848/1725] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:39 2022] POST /identity/v3/projects => generated 391 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:40.391527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.395833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.441623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-875c330e-1df1-4406-8c8e-e0a3e98093f2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:40.445795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-875c330e-1df1-4406-8c8e-e0a3e98093f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.634869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8ec2-00b1-4ed5-91b9-28bb24eea6db admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.646264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 878/1726] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:40.647957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e9263-f4de-42a0-bd0c-298f0a17f706 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.648100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e9263-f4de-42a0-bd0c-298f0a17f706 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.648347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e9263-f4de-42a0-bd0c-298f0a17f706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.655743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.657174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.657634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.658052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.658895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.660622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.682893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 849/1727] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:40.683898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.684669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f3e9263-f4de-42a0-bd0c-298f0a17f706 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:40.692576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f3e9263-f4de-42a0-bd0c-298f0a17f706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.698580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.717743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 879/1728] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:40.720464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.721991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.722280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.761822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca8786-975e-4ae8-97bd-a1aefff63e4a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.782992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:40.787650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.791047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.793295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.795491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.796238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.798158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.803353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0811b46-0652-418b-bdd2-b83288a63df1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.817667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 880/1729] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:40.818709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.833008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7492b-0e14-4243-88e4-06e3a795f69e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:40.842008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:40.842955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.843337 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.843509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] PATH_INFO: `/v3/projects/9c0483f042394228a976f25159820340/users/271a3f57a2f34b9caa623fc64abb5119/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.850155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 850/1730] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:40.854032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:40.857507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:40.858531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:40.873167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0483f042394228a976f25159820340, user_id=271a3f57a2f34b9caa623fc64abb5119, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:40.876293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:40.908692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 881/1731] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:40 2022] PUT /identity/v3/projects/9c0483f042394228a976f25159820340/users/271a3f57a2f34b9caa623fc64abb5119/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:40.909986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:40.929709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:40.937213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:40.976306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:41.003140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f669e8-a37b-4665-8b3f-b38604cd8b3c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.008727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.009582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.009895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.010181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.011056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:41.011455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.014047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.053392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 882/1732] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:41.054289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.070229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecff38-75bd-48c7-82e9-75e150e70e51 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.072705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:41.078677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.080396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.080897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.082359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.083393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.086371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.102869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 883/1733] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:41.104159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.110026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 851/1734] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:41.111095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.116872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78334f13-03ef-4d0b-8677-2e4afe1fc4da admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.123782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.124424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92578a-19c9-40ee-8957-6bb782151c2d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.125720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.126212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.126700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.127811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.131165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.138941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.139638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.140124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.140410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.141834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.144720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.155602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 884/1735] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:41.156971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.183465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35460d4-9959-42dc-be44-3b80ad246a7c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.191610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.192791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.193565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.194012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] PATH_INFO: `/v3/projects/2203d800fe5a4817950b059738ce8fd3/users/52568ea7db1a474bb7701c1bc52fcc1f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.228144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 852/1736] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:41 2022] POST /identity/v3/projects => generated 407 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:41.232335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2203d800fe5a4817950b059738ce8fd3, user_id=52568ea7db1a474bb7701c1bc52fcc1f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.246030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.251134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.294649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 885/1737] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:41 2022] PUT /identity/v3/projects/2203d800fe5a4817950b059738ce8fd3/users/52568ea7db1a474bb7701c1bc52fcc1f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:41.302009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.496691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e63361-9eca-44c6-99a9-e75737220d9b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.509390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292924be-e86c-4354-b72b-1a5bd538dee9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.511290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.523994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.524417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.524890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.526084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:41.526704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.527407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.528737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.530080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.530550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.530983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] PATH_INFO: `/v3/projects/9c0483f042394228a976f25159820340/users/271a3f57a2f34b9caa623fc64abb5119/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.535234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0483f042394228a976f25159820340, user_id=271a3f57a2f34b9caa623fc64abb5119, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.538675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.568170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 853/1738] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:41.568357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 886/1739] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:41 2022] PUT /identity/v3/projects/9c0483f042394228a976f25159820340/users/271a3f57a2f34b9caa623fc64abb5119/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:41.569223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.580320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.626275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87349cb5-1e41-431a-b6c9-a86d282a3551 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.782346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7c711-956d-4ca8-907b-6d04ded66929 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.782607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.783520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.783961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.784334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.785242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.787501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffb296-2085-4b87-9393-3fa428b62eaa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.805293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.806393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.806685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.806971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.807977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.809791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.832876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 887/1740] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:41.836203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:41.850029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 854/1741] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:41 2022] POST /identity/v3/users => generated 304 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:41.851640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.851939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.852225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.866287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9249b-0a86-4c47-94dc-d4353a6ae4c8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:41.903105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:41.903821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:41.904115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:41.904418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:41.905123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:41.918697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:41.971191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] MFA Rules not processed for user `271a3f57a2f34b9caa623fc64abb5119`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:41.972678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:41.981795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:42.008480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 888/1742] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:41 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:42.009396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.100147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:42.151925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 855/1743] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:42.170702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.182225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3060e0-86e4-4e87-83eb-fb39a1110df4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.188965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c78ad3-69c6-40e0-b63f-37f1a651dfee None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.196392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.197614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.198063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.198521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] PATH_INFO: `/v3/projects/2203d800fe5a4817950b059738ce8fd3/users/52568ea7db1a474bb7701c1bc52fcc1f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.199704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.201611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.201900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.202178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.203744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2203d800fe5a4817950b059738ce8fd3, user_id=52568ea7db1a474bb7701c1bc52fcc1f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.204286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.210017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.210654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.240933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 856/1744] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:42.240933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.247743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 889/1745] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:42 2022] PUT /identity/v3/projects/2203d800fe5a4817950b059738ce8fd3/users/52568ea7db1a474bb7701c1bc52fcc1f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:42.256726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a657773-6c21-4577-9102-99a820772256 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.260122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.265326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:42.387458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.388438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.388919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.389207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.390135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.393141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.431712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2ee45-384f-4e8c-9837-dd396a8f96e3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.436451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.437642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.440049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.440629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.444399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 857/1746] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:42 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:42.445995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.447436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:42.486598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34597-f57c-4ef2-9c8e-74c8cde9e64f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.501884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.502841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.503302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.503742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.504661 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.509708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715f3b7-6d4e-4f54-8b82-87c92f1a8eff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.530759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 858/1747] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:42.533902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.534479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.534861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.572070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] MFA Rules not processed for user `52568ea7db1a474bb7701c1bc52fcc1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:42.573902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:42.618904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.620451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.652225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 890/1748] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:42.653680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.675464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147012-3da7-417a-afe1-4c1118cb31a6 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.699978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.700676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.701658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.701945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.702689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.704399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.759184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 891/1749] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:42.773004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.830205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 859/1750] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:42.830728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33508-c320-453f-81c6-28daeda3ec24 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.836026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.865027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.866320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.866608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.866888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] PATH_INFO: `/v3/projects/9de2c5b2f9a14f1799c00643bc1bdec2/users/10b84e42d8094b0ba2c18611b35688ad/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.879178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79f10-5e0b-4e63-9a29-204fb2b64dff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:42.900935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:42.903644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:42.903958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:42.905816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:42.907182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.912236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.940121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de2c5b2f9a14f1799c00643bc1bdec2, user_id=10b84e42d8094b0ba2c18611b35688ad, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:42.952955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 860/1751] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:42.953837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:42.955318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:42.988473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271ccdb-6505-45bd-b157-6efed5b3ccb5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:43.003193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 892/1752] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:42 2022] PUT /identity/v3/projects/9de2c5b2f9a14f1799c00643bc1bdec2/users/10b84e42d8094b0ba2c18611b35688ad/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:43.005099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:43.043920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:43.076122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:43.197566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acd161-9603-420d-8578-b3718988349f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:43.198558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd161-9603-420d-8578-b3718988349f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.199023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd161-9603-420d-8578-b3718988349f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.204239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acd161-9603-420d-8578-b3718988349f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.214874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9eb61-1887-470d-998a-3c6606f179f8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:43.234378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:43.235393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.235697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.235990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] PATH_INFO: `/v3/projects/d7112897d6524ad48391981ab7252359/users/1b4c9cf34d1f44d191a93d3eb275879f/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.270182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7112897d6524ad48391981ab7252359, user_id=1b4c9cf34d1f44d191a93d3eb275879f, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:43.272929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020547fb-9d51-494c-a612-10ad7cc3570c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:43.311815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 893/1753] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:43 2022] PUT /identity/v3/projects/d7112897d6524ad48391981ab7252359/users/1b4c9cf34d1f44d191a93d3eb275879f/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:43.311815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170963d1-6870-4e57-a363-568e2012e276 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.311815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170963d1-6870-4e57-a363-568e2012e276 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.311815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170963d1-6870-4e57-a363-568e2012e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.356013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-170963d1-6870-4e57-a363-568e2012e276 None None] MFA Rules not processed for user `10b84e42d8094b0ba2c18611b35688ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:43.357218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-170963d1-6870-4e57-a363-568e2012e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.365719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29acd161-9603-420d-8578-b3718988349f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:43.368563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29acd161-9603-420d-8578-b3718988349f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:43.381965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 861/1754] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:43.395723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.396018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.396353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.441188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] MFA Rules not processed for user `3753498f22d5443ab0c01473c91a263c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:43.442455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.480059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 894/1755] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:43.483765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.484122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.484532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.550433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] MFA Rules not processed for user `1b4c9cf34d1f44d191a93d3eb275879f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:43.553827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.605112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 862/1756] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:43.605112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:43.720156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 895/1757] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:43.721201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:43.767627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:43.858129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85002c8-d3ab-4a77-9aa4-3e3d5cb57bb1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:43.866834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:43.868349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.868639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.869367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.872912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:43.932463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659e3bd-3295-43c3-9144-d22fdd6bd995 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:43.951622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:43.954260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:43.954612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:43.954932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:43.955777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:43.959647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:43.977890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:44.037144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 896/1758] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:43 2022] POST /identity/v3/projects => generated 409 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:44.039542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.049408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.065202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.066681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de3d5-ff47-4a9d-ad6c-c43e1a50c791 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.094696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 863/1759] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:44.096076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.096369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.096645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.214698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:44.215984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.227813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2605c90-2f85-421e-aa17-09845b6955dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:44.236144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:44.236904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.237483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.237802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.321244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 864/1760] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:44.322155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.389828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.399586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.441318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 897/1761] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:44.449861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.451534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.505187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19703b34-1acc-4cd3-8d54-5c346e335301 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:44.510211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:44.511069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.514966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.515558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.516974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.519424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.556402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 865/1762] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:44 2022] POST /identity/v3/users => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:44.560272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:44.562816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.591339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6143c7-f4d4-4495-b57e-2ee75c81e7d6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:44.603515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:44.605629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.606554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.607244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.609817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.615423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.641398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 866/1763] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:44.643589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.660943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.684969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c0c2f-4aee-4b8d-b16c-53f06b2df8f3 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:44.704160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:44.706090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.706373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.710823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] PATH_INFO: `/v3/users/d0f22ba036b84cb194732c02570e8568` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.716446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f22ba036b84cb194732c02570e8568)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.717003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:44.723780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.859429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f22ba036b84cb194732c02570e8568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:44.876396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f22ba036b84cb194732c02570e8568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:44.888503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd09f-f960-4b95-92a1-f47d8a75dab4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:44.896076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:44.896942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.897207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f22ba036b84cb194732c02570e8568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:44.897315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.897753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.898714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f22ba036b84cb194732c02570e8568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:44.900607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.908552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f22ba036b84cb194732c02570e8568'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:44.915012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:44.916875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067ed8d-39ab-4f83-a76d-809f44db9aa1 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:44.918013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 898/1764] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:44 2022] DELETE /identity/v3/users/d0f22ba036b84cb194732c02570e8568 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:44.919237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:44.922222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:44.935755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 867/1765] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:32:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:32:44.939447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:44.940006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:44.940344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:44.986370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:44.987602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.162655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbee88-77b3-4a0d-ba7e-c5f2f3b67c3f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:45.162889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 868/1766] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:45.191924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:45.192824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.193134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.193549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] PATH_INFO: `/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57/users/d44757e400bd4a1a96341a832c41fddd/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.228183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b91d7e3fcf4ca493ec9224dd3e1b57, user_id=d44757e400bd4a1a96341a832c41fddd, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:45.233655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:45.268042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 899/1767] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:44 2022] PUT /identity/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57/users/d44757e400bd4a1a96341a832c41fddd/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:45.272407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:45.272695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:45.275025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.295165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ba5b3-0c9f-45a4-a752-9dec112dde80 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:45.453759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:45.543231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8d374-6f76-42ba-934a-ba6148061572 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:45.563880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:45.564663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.564947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.572865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.584542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:45.585239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.593518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.594374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.595173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:45.609161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:45.620202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:45.633797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b5fb-0cf3-40b8-9a63-e15db649a8b6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:45.678689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 869/1768] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:45.679777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:45.688038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 900/1769] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:45.705469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98bf44-5424-4fc4-8ea8-0ed20f7cc31b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:45.719202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:45.722737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.723712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.724629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] PATH_INFO: `/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57/users/d44757e400bd4a1a96341a832c41fddd/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.731268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b91d7e3fcf4ca493ec9224dd3e1b57, user_id=d44757e400bd4a1a96341a832c41fddd, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:45.741211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:45.791062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 870/1770] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:45 2022] PUT /identity/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57/users/d44757e400bd4a1a96341a832c41fddd/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:45.804165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:45.804480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:45.804733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:45.832330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:45.841210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.928262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] MFA Rules not processed for user `d44757e400bd4a1a96341a832c41fddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:45.931685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:45.936159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:46.050175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579c67-1556-4fb6-836d-10b6f2d71416 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:46.067644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:46.069166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.069483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.070882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] PATH_INFO: `/v3/users/1b4c9cf34d1f44d191a93d3eb275879f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.072568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4c9cf34d1f44d191a93d3eb275879f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:46.075675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:46.134221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 901/1771] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:46.155735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:46.159160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.259396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4c9cf34d1f44d191a93d3eb275879f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:46.284311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4c9cf34d1f44d191a93d3eb275879f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:46.285299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4c9cf34d1f44d191a93d3eb275879f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:46.319821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4c9cf34d1f44d191a93d3eb275879f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:46.338831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10590cf7-707c-4fe3-ac43-e0f452426e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4c9cf34d1f44d191a93d3eb275879f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:46.354465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 871/1772] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:45 2022] DELETE /identity/v3/users/1b4c9cf34d1f44d191a93d3eb275879f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:46.359286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.359654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.359858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.418046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffea22-e8a6-470d-afee-d80c299fa7ed None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:46.440320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:46.449427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.625600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:46.626433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.626826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.627466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.643559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 872/1773] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:46.647117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.747139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:46.785711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:46.792821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:46.812191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:46.813940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769b84-c1b1-433c-9b11-ae6c94bb5212 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:46.838823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 902/1774] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:46.905216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:46.957563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b2fb7-03d3-41d0-85f5-a0afb32a0bb9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:46.964576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.965032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.965315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:46.984556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:46.986559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:46.997849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:46.998959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.034158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:47.041062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:47.044732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.063777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:47.082052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 903/1775] 10.222.0.76 () {58 vars in 1015 bytes} [Tue Jul 12 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:47.098679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 873/1776] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:47.103587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:47.147394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577bcc0-6fc9-465e-b722-95e147c633b3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:47.168566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:47.176053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:47.177382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:47.179701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.227012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:47.227012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:47.253434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 904/1777] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:47.312484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:47.314825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.338615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a3e7e-076b-4dd1-9a1f-154b2366f7f7 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:47.349499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:47.354296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:47.355384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:47.356552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.401260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:47.410451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:47.477421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 874/1778] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:47.506955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:47.509095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.537840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc426-d68d-44a6-98ef-7f728698f472 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:47.593285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:47.594619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:47.595021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:47.595643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.604354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.827241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:47.831235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:47.864074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9a7e-8875-4cca-8008-6ac612976732 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:47.895426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:47.896916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:47.899193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:47.901884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] PATH_INFO: `/v3/projects/f0c7a92dfcc847e086b94b2875d109db` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:47.904337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c7a92dfcc847e086b94b2875d109db)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:47.910684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:48.038774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:48.038774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:48.090881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 905/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:48.101890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c7a92dfcc847e086b94b2875d109db', 'request_id': 'req-238bf996-53b0-448f-b0fc-7c621011fe12'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.144822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c7a92dfcc847e086b94b2875d109db', 'request_id': 'req-238bf996-53b0-448f-b0fc-7c621011fe12'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.167034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c7a92dfcc847e086b94b2875d109db', 'request_id': 'req-238bf996-53b0-448f-b0fc-7c621011fe12'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.213947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] The token cache is being invalidate because project f0c7a92dfcc847e086b94b2875d109db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:48.214234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.218784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.233084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 875/1780] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:47 2022] DELETE /identity/v3/projects/f0c7a92dfcc847e086b94b2875d109db => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:48.234854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:48.253624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402a0fa-24d9-453b-8680-05f7d061f754 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:48.491073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:48.496301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:48.496891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:48.497271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] PATH_INFO: `/v3/users/e6e54193f298417faa2f187f945c65f1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:48.499528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e54193f298417faa2f187f945c65f1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:48.506177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:48.676850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e54193f298417faa2f187f945c65f1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.691330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:48.706451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e54193f298417faa2f187f945c65f1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.790522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e54193f298417faa2f187f945c65f1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.793562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e54193f298417faa2f187f945c65f1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.861376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e79e8173-cb7a-4b36-8c71-a962d5ce0d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e54193f298417faa2f187f945c65f1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:48.886032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 906/1781] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:48 2022] DELETE /identity/v3/users/e6e54193f298417faa2f187f945c65f1 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:48.987813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:48.989859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:49.013831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf996-53b0-448f-b0fc-7c621011fe12 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:49.218454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:49.222278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.222816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.223340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] PATH_INFO: `/v3/projects/d7112897d6524ad48391981ab7252359` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.225016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7112897d6524ad48391981ab7252359)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:49.227943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:49.327636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7112897d6524ad48391981ab7252359', 'request_id': 'req-03065f0e-756e-4146-a95c-d435c7fba687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:49.332549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:49.348348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7112897d6524ad48391981ab7252359', 'request_id': 'req-03065f0e-756e-4146-a95c-d435c7fba687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:49.368569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7112897d6524ad48391981ab7252359', 'request_id': 'req-03065f0e-756e-4146-a95c-d435c7fba687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:49.435880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] The token cache is being invalidate because project d7112897d6524ad48391981ab7252359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:49.435880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:49.435880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:49.435880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 876/1782] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:48 2022] DELETE /identity/v3/projects/d7112897d6524ad48391981ab7252359 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:49.439116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcba6-09d7-4b18-8f4c-5d89ec717764 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.439571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcba6-09d7-4b18-8f4c-5d89ec717764 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.442314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fcba6-09d7-4b18-8f4c-5d89ec717764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.503595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-328fcba6-09d7-4b18-8f4c-5d89ec717764 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:49.508062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-328fcba6-09d7-4b18-8f4c-5d89ec717764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:49.553762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:49.556868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:49.670808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03065f0e-756e-4146-a95c-d435c7fba687 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:49.693506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:49.694195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.694795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.695085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.697970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:49.736367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 907/1783] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:49.744504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.745490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.745490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.780676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:49.783763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:49.806840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 908/1784] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:49.811571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:49.825612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00197b7-a010-45e2-b925-f0e59d5d7d1f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:49.833251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:49.834231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.834906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.835287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.836136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:49.836660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:49.850736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:49.877088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 909/1785] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:32:49.878125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:49.878495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:49.880213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd927f-3db6-457a-b9f6-46e7ea139110 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:49.906746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7bbd-8df8-483f-a6d0-998b938ffad2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:49.911905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:49.912582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:49.912864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:49.913157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:49.913884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:49.915957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884d53b-bbe4-4b72-97bd-84cc9bb1fa6a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:49.933859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 877/1786] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:50.043040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98175a2-421a-4c2c-8595-e700e684c4bf None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.043040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98175a2-421a-4c2c-8595-e700e684c4bf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.043786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98175a2-421a-4c2c-8595-e700e684c4bf None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.045644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 878/1787] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:32:50.059572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 910/1788] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:49 2022] POST /identity/v3/projects => generated 371 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:50.064820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98175a2-421a-4c2c-8595-e700e684c4bf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.094299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.094627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.094911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.158468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:50.160310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:50.319281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98175a2-421a-4c2c-8595-e700e684c4bf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.332358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.333220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.333538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.333946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.334659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.336750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.379017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 911/1789] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:32:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:50.397464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 879/1790] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:50 2022] POST /identity/v3/users => generated 286 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:50.401599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.471676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806476f1-d337-4648-9616-7a1486cd2bb5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.491066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.491847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.492147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.492495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.493175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.494815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.525612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.527954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:50.586197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 912/1791] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:50.587635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.596238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3771-33ad-4979-9ee5-fcf26fef77d3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.615806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d22d1-d986-4875-95b6-05c7b6192337 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.637243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.638600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.639047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.641429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.643191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.649041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48094-5738-43e8-97aa-423a84aede12 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.655881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:50.663302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.664866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.668072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.669626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.709629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 913/1792] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:50.712747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.717400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.718065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.728958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 914/1793] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:32:50 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:32:50.730075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.739903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:50.741241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.753965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.787587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df276f-8f9d-4d14-99c2-89e963f57be3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.803530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.811823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.814737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.816246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] PATH_INFO: `/v3/users/ac00fffa0ac04728a442234dc8736f15` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.821057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac00fffa0ac04728a442234dc8736f15)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.831525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.846459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 880/1794] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:32:50.848495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:50.896087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5f1c-3dc4-4c76-9f3b-ad326834e5aa service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:50.918497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:50.921060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:50.922813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:50.924017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] PATH_INFO: `/v3/projects/7077f73e40f84278b2ce7a31326e8698/users/32a464377b714c3eb5e59d7377ce2413/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:50.932900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:50.971333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7077f73e40f84278b2ce7a31326e8698, user_id=32a464377b714c3eb5e59d7377ce2413, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:50.978881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4ce22-cedc-4e66-b3b7-7e6997dafeee admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:50.996924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac00fffa0ac04728a442234dc8736f15'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:51.013051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac00fffa0ac04728a442234dc8736f15'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:51.028685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 881/1795] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:50 2022] PUT /identity/v3/projects/7077f73e40f84278b2ce7a31326e8698/users/32a464377b714c3eb5e59d7377ce2413/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:51.028685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390713c5-c312-43a5-be85-1a68cf86dbc0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.028685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390713c5-c312-43a5-be85-1a68cf86dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.028685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390713c5-c312-43a5-be85-1a68cf86dbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.050264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac00fffa0ac04728a442234dc8736f15'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:51.052353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac00fffa0ac04728a442234dc8736f15'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:51.087746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-59b408b7-f3b3-451d-baf0-25f490d56f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac00fffa0ac04728a442234dc8736f15'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:51.090131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-390713c5-c312-43a5-be85-1a68cf86dbc0 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:51.096785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-390713c5-c312-43a5-be85-1a68cf86dbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.111415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 915/1796] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:50 2022] DELETE /identity/v3/users/ac00fffa0ac04728a442234dc8736f15 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:51.113183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.113657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.114256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.210359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] MFA Rules not processed for user `32a464377b714c3eb5e59d7377ce2413`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:51.211917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.292277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 882/1797] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:51.293652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.293944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.294226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.338810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:51.345692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.352995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 916/1798] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:51.357677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:51.359736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.544529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 883/1799] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:51.569591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce7900-78a4-4c8f-b102-dae281a5dcfa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:51.583095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:51.584131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.589473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.590049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.591525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:51.592372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:51.594509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:51.594655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.605405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:51.643910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 917/1800] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:51 2022] POST /identity/v3/projects => generated 371 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:32:51.647713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:51.721933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b7e98-0496-4fdd-b394-f995d1c3dfd8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:51.793458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:51.814988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fe174-749e-4bc1-bdb8-35498aa0a01b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:51.827572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:51.829145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.830745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.831279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.832443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:51.835255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:51.877533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:51.878243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.881144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.881680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 918/1801] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:51 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:51.882799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:51.885097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.885515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:51.916858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:51.918912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:51.948791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 884/1802] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:51.950380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:51.971677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5bf16-0be5-4c32-862f-7bc0bad5ded5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:51.981272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:51.988019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:51.988519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:51.988949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:51.990443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:51.992382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.016710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 885/1803] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:52.021620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.044625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c42737-8a8f-4c92-a932-340d8ab0fd36 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.060216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.061277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.061766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.068482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] PATH_INFO: `/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb/users/c73df3b2ba784620a0cab070867e454a/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.077096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca28c-534b-4df5-a6b4-1e9f8252834b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.084099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.084796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.085118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.085439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.097054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.098874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.104058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a96ab5e166045a5975c0f2ecb0639bb, user_id=c73df3b2ba784620a0cab070867e454a, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.110110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.117241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 919/1804] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:52.143344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 886/1805] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:52 2022] PUT /identity/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb/users/c73df3b2ba784620a0cab070867e454a/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:52.160874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.185367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45a11f-39ce-4e7c-bc03-efb065213f32 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.236769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.239038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:52.428697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.428697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.428697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.428697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.429396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.434726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.441720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:52.451991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c2c23-2764-4c92-8bbc-1faa079037d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.473532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 920/1806] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:52.474796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.475484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.476434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.477066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.477689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] PATH_INFO: `/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.479881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3cca85b7c2f41da8c7289f1c673c8cb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.483625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.519384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6ce6-ad02-4be0-9ac9-489c7a7e8bb9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.542312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.543802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.544287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.544786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] PATH_INFO: `/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb/users/c73df3b2ba784620a0cab070867e454a/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.555829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a96ab5e166045a5975c0f2ecb0639bb, user_id=c73df3b2ba784620a0cab070867e454a, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.570009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32c895-5cea-4c71-8149-864a98a23207 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.590058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 921/1807] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:52 2022] PUT /identity/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb/users/c73df3b2ba784620a0cab070867e454a/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:52.601897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.602192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.602476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.605794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3cca85b7c2f41da8c7289f1c673c8cb', 'request_id': 'req-9960edfe-3780-4cf1-9b55-eb1ba17e345c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:52.632974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3cca85b7c2f41da8c7289f1c673c8cb', 'request_id': 'req-9960edfe-3780-4cf1-9b55-eb1ba17e345c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:52.633900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] MFA Rules not processed for user `c73df3b2ba784620a0cab070867e454a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:52.635140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:52.646375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3cca85b7c2f41da8c7289f1c673c8cb', 'request_id': 'req-9960edfe-3780-4cf1-9b55-eb1ba17e345c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:52.664476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] The token cache is being invalidate because project c3cca85b7c2f41da8c7289f1c673c8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:52.664827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:52.666439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9960edfe-3780-4cf1-9b55-eb1ba17e345c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:52.673241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 887/1808] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:52 2022] DELETE /identity/v3/projects/c3cca85b7c2f41da8c7289f1c673c8cb => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:52.674956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c31c5-840b-49dc-bf7a-f5210345e133 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.675258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c31c5-840b-49dc-bf7a-f5210345e133 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.675573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c31c5-840b-49dc-bf7a-f5210345e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.698699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cc5c31c5-840b-49dc-bf7a-f5210345e133 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:52.701491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc5c31c5-840b-49dc-bf7a-f5210345e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:52.731867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 922/1809] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:52.732674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:52.743667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.745102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:52.779843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 888/1810] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:52.789697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.789998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.790279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.810248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:52.811371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:52.820639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe86ee-ac7a-4a2f-a981-03f97a1449cb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.824699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 889/1811] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:52.825637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.827152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.828853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.829378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.830083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.835828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aeb49-0c11-4955-8ace-52952ec117c0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.841486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.842313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.842736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.843299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.844204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:52.844633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.844761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.846444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.846563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.864621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 890/1812] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:52.870161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.878032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 923/1813] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:32:52.888022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee84ecb-31e4-4051-bcd8-1c528a7b9f33 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:52.896300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:52.897918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:52.898562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:52.898893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:52.900787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:52.902571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:52.933836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 891/1814] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:52 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:52.938698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:52.953846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9acda-dbd0-4863-9e50-e6727397883d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.043143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.044126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.044528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.044906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.046598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.048491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.073135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.077168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.079672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.082701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 924/1815] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:52 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:53.086662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.104460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305691d2-1fa7-420a-9da7-e5a4d4c0a089 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.116618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.117928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.118225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.118511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.119267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.119650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa6eec-684e-4b0b-bfc3-9527164fff2f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.121533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.138073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 925/1816] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:32:53.138905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.153480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4da50-f97d-45a0-8836-f90af7165a5f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.154652 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.163834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.164160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.164443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] PATH_INFO: `/v3/projects/04a8ce3db02b40a9abaaf6d69c6a2309` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.165840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.166746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.167125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.167771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.168803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a8ce3db02b40a9abaaf6d69c6a2309)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.169253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.171376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.173012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.184920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 926/1817] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:53.189632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.201185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe00c-3efd-4684-88dd-3b559e105846 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.213258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.214341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.214754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.215069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] PATH_INFO: `/v3/projects/ee85aa5a77d8468ab6223adea9777afe/users/247eb142205647a89d89d4fbe68aca1d/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.241523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee85aa5a77d8468ab6223adea9777afe, user_id=247eb142205647a89d89d4fbe68aca1d, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.253828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b03a7-8cc6-4384-8edd-6b7170bb16db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.263503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a8ce3db02b40a9abaaf6d69c6a2309', 'request_id': 'req-a524a826-185f-4ba0-ad59-6df8f3202f73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.282530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a8ce3db02b40a9abaaf6d69c6a2309', 'request_id': 'req-a524a826-185f-4ba0-ad59-6df8f3202f73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.285384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 927/1818] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:53 2022] PUT /identity/v3/projects/ee85aa5a77d8468ab6223adea9777afe/users/247eb142205647a89d89d4fbe68aca1d/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:53.298557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.299281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.299722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.304263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a8ce3db02b40a9abaaf6d69c6a2309', 'request_id': 'req-a524a826-185f-4ba0-ad59-6df8f3202f73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.321397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] The token cache is being invalidate because project 04a8ce3db02b40a9abaaf6d69c6a2309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:53.321768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.322516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.327310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 892/1819] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:53 2022] DELETE /identity/v3/projects/04a8ce3db02b40a9abaaf6d69c6a2309 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:53.330536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.332027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.345974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] MFA Rules not processed for user `247eb142205647a89d89d4fbe68aca1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:53.345974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.430992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524a826-185f-4ba0-ad59-6df8f3202f73 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.449581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.457521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 928/1820] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:53.462368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.465486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.518704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.519557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.519971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.520255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] PATH_INFO: `/v3/users/52568ea7db1a474bb7701c1bc52fcc1f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.523838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52568ea7db1a474bb7701c1bc52fcc1f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.523838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.543540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516e738-b11c-4d48-9a82-dcc1d7f6ffff None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.550928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.551979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.552400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.552924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.553669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.556031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.581450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52568ea7db1a474bb7701c1bc52fcc1f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.584408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 929/1821] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:53 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:32:53.585606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.587312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.592130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52568ea7db1a474bb7701c1bc52fcc1f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.592932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52568ea7db1a474bb7701c1bc52fcc1f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.605516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52568ea7db1a474bb7701c1bc52fcc1f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.612966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52568ea7db1a474bb7701c1bc52fcc1f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.618976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 893/1822] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:53 2022] DELETE /identity/v3/users/52568ea7db1a474bb7701c1bc52fcc1f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:53.619724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.621076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.626720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.670656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc126032-1e4c-4382-a790-0f4f754eeaba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.723702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de4f40-66c8-4e29-adf0-e8f6bc734f27 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.728874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.729584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.729884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.730219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.731081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.733142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.738352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.739509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.740119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.740939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] PATH_INFO: `/v3/users/32a464377b714c3eb5e59d7377ce2413` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.742531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a464377b714c3eb5e59d7377ce2413)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.744767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.763579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 894/1823] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:53 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:53.769702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.781714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc2920-810f-47c9-89e3-147dfeaa8b5c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.787753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.788431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.788817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.789103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.789964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.791483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.803810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 895/1824] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:32:53.807560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.812436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a464377b714c3eb5e59d7377ce2413'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.818742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3d553-2763-4d3e-8393-cf8fad1e4f24 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:53.822234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a464377b714c3eb5e59d7377ce2413'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.824397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:53.825399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.825761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.826627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] PATH_INFO: `/v3/projects/31710c3ca48746c1b73142795b5f15d0/users/0355504376b94d3db5a1affa8aa67536/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.837098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a464377b714c3eb5e59d7377ce2413'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.837960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a464377b714c3eb5e59d7377ce2413'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.842450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.846680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31710c3ca48746c1b73142795b5f15d0, user_id=0355504376b94d3db5a1affa8aa67536, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:53.847278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5ace57cb-c92f-4dc6-8fe0-42c02fd49eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a464377b714c3eb5e59d7377ce2413'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:53.850942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:53.854225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 930/1825] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:53 2022] DELETE /identity/v3/users/32a464377b714c3eb5e59d7377ce2413 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:53.858974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:53.859435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:53.859719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:53.870306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 896/1826] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:53 2022] PUT /identity/v3/projects/31710c3ca48746c1b73142795b5f15d0/users/0355504376b94d3db5a1affa8aa67536/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:53.871445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.874239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.881501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:53.882973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:53.952377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:53.978678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 931/1827] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:32:53.980081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:53.987967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.000841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce829f38-6168-4d47-8acc-d915c06ef86b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.004369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa9ef-bdb6-43f3-a1c4-317fbe491147 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.007332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.009207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.011066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.012084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] PATH_INFO: `/v3/projects/2203d800fe5a4817950b059738ce8fd3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.016184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2203d800fe5a4817950b059738ce8fd3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.018023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.019013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.019399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.019781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.022983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.025181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.036621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.049013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 897/1828] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:54.049518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.052744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.160650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808712f5-350e-4c09-920d-7a7b654ca8ce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.164401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2203d800fe5a4817950b059738ce8fd3', 'request_id': 'req-b68de3ff-8276-493c-95ae-e7ca318442ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:54.175461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2203d800fe5a4817950b059738ce8fd3', 'request_id': 'req-b68de3ff-8276-493c-95ae-e7ca318442ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:54.183733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2203d800fe5a4817950b059738ce8fd3', 'request_id': 'req-b68de3ff-8276-493c-95ae-e7ca318442ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:54.197368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] The token cache is being invalidate because project 2203d800fe5a4817950b059738ce8fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:54.197618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:54.198406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:54.205232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 932/1829] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:53 2022] DELETE /identity/v3/projects/2203d800fe5a4817950b059738ce8fd3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:54.206629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.208400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.238459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.239650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.240171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.240690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.243092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.340663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de3ff-8276-493c-95ae-e7ca318442ab admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.347634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.348677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.349098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.349501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] PATH_INFO: `/v3/projects/31710c3ca48746c1b73142795b5f15d0/users/0355504376b94d3db5a1affa8aa67536/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.352888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31710c3ca48746c1b73142795b5f15d0, user_id=0355504376b94d3db5a1affa8aa67536, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.356027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df43fb-3c4d-43ec-9c02-6a390d683728 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.364649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:54.375236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.376948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 933/1830] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:54 2022] PUT /identity/v3/projects/31710c3ca48746c1b73142795b5f15d0/users/0355504376b94d3db5a1affa8aa67536/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:54.379108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2f4d-4166-4611-88dc-acf46eb9582d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.379669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2f4d-4166-4611-88dc-acf46eb9582d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.379784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f72c-173a-40a7-87a1-84751eef0f95 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.380367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb2f4d-4166-4611-88dc-acf46eb9582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.398263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 898/1831] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:54.413359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.415537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.415852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.426343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-afdb2f4d-4166-4611-88dc-acf46eb9582d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:54.427581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-afdb2f4d-4166-4611-88dc-acf46eb9582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.464689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] MFA Rules not processed for user `0355504376b94d3db5a1affa8aa67536`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:54.466433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.585135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 934/1832] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:54.594354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.596388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.597366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.608702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 899/1833] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:54.624409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.629174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.641708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:54.645370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.672727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 935/1834] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:54.674766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.701128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c20f-8c53-4873-91fc-8c28198d399f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.710607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.711297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.711584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.711866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.712679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:54.713050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.716310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.718325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:54.771662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 936/1835] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:54.771877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e11ef-0482-4a08-b107-cb6e36342fbc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.772874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.775812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.776388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.777719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.777719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.779951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.785920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac2e97-8da0-4665-902f-11393df7f69e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.790909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.791843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.792250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.792628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.793466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.795844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.851133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 937/1836] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:54 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:54.856161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.891480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.893166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.902838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 900/1837] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:54.933311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:54.934792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:54.947627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5c10-78e0-491a-8b44-5c6e6dce8eec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.951757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc8dc-935a-45fa-9071-d2ab428f9671 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:54.954568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.955459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.955886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.956397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.957448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.960889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:54.960889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:54.960889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:54.960889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] PATH_INFO: `/v3/projects/7077f73e40f84278b2ce7a31326e8698` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:54.962316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7077f73e40f84278b2ce7a31326e8698)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:54.965402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:54.967703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.010775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 938/1838] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:54 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:55.015842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:55.022098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:55.040801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d457-bf11-412a-96f1-391dcd1d1fae admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:55.054428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7077f73e40f84278b2ce7a31326e8698', 'request_id': 'req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.069153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7077f73e40f84278b2ce7a31326e8698', 'request_id': 'req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.077466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7077f73e40f84278b2ce7a31326e8698', 'request_id': 'req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.089999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] The token cache is being invalidate because project 7077f73e40f84278b2ce7a31326e8698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:55.091216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.092061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.097090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 901/1839] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:54 2022] DELETE /identity/v3/projects/7077f73e40f84278b2ce7a31326e8698 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:55.098385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:55.100817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:55.112583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:55.113438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.113777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.114125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.114938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:55.116801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.140836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 939/1840] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:55.142025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:55.148815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:55.173292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212602-950c-407d-b5c6-31392285d637 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:55.186959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:55.188272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.188703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.188813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.189922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:55.192300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.203223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 940/1841] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:55.207743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:55.244873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d34e54-20fb-4e90-ab72-260738e02975 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:55.259857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:55.261784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.262330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.263040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] PATH_INFO: `/v3/projects/7c91d4aa019a4b19811da935b7347d83/users/a0601704fdd847188095e13e2632edf4/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.277114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0f78-2ce7-4c14-8099-2e29d75bd652 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:55.286297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c91d4aa019a4b19811da935b7347d83, user_id=a0601704fdd847188095e13e2632edf4, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:55.289379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75301e3d-64a5-4efc-96a5-e6e9e4ee8eda admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.298625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:55.299677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.300139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.300468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] PATH_INFO: `/v3/users/c73df3b2ba784620a0cab070867e454a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.303191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73df3b2ba784620a0cab070867e454a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:55.305992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.331461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 941/1842] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:55 2022] PUT /identity/v3/projects/7c91d4aa019a4b19811da935b7347d83/users/a0601704fdd847188095e13e2632edf4/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:55.335345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.335827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.336286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.422527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] MFA Rules not processed for user `a0601704fdd847188095e13e2632edf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:55.423320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:55.435918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73df3b2ba784620a0cab070867e454a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.447673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:55.453391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73df3b2ba784620a0cab070867e454a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.463315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73df3b2ba784620a0cab070867e454a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.515604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73df3b2ba784620a0cab070867e454a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.542904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73df3b2ba784620a0cab070867e454a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:55.550863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 902/1843] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:32:55 2022] DELETE /identity/v3/users/c73df3b2ba784620a0cab070867e454a => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:55.667216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:55.668929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 942/1844] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:55.669231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:55.771296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:55.823712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff5eb5-a384-4238-a66c-bb15d8485e53 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:55.830246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:55.831058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:55.831444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:55.831844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:55.832632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:55.836745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:55.904115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 903/1845] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:55 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:55.908678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.116804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f932097-3f16-49c5-8cc7-4a24035b65b7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.268567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.268567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.268567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.268567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.269050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.269050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.332586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.335387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 943/1846] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:55 2022] POST /identity/v3/users => generated 298 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:56.357219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a896b9-e986-421e-837d-efcca64d298e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.376816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.379948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.381407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.383261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.385181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.391000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.420862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 904/1847] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:32:56.425845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.440009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70600b-0ad6-49b8-87d2-a9c5d14b6592 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.449318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.454714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.455634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.456331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] PATH_INFO: `/v3/projects/2ddcad57c19049178d00e9aee13a97f6/users/66a6bcc79a3e4b528b524ddb40bc1300/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.509061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ddcad57c19049178d00e9aee13a97f6, user_id=66a6bcc79a3e4b528b524ddb40bc1300, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.517225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.549902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.550740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 944/1848] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:56 2022] PUT /identity/v3/projects/2ddcad57c19049178d00e9aee13a97f6/users/66a6bcc79a3e4b528b524ddb40bc1300/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:56.564056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8eba4b-c055-471e-bf07-c3c85041968c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.681236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.682115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.682486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.683619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.686235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.688420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d782eb-474b-4975-abe4-6de098eb399d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.704674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 905/1849] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:56.709264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:56.728849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ff113-ea2f-458b-8cbf-012e0f92f4e9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:56.741197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:56.742432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.742812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.743185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] PATH_INFO: `/v3/projects/2ddcad57c19049178d00e9aee13a97f6/users/66a6bcc79a3e4b528b524ddb40bc1300/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.750814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ddcad57c19049178d00e9aee13a97f6, user_id=66a6bcc79a3e4b528b524ddb40bc1300, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:56.768957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:56.819328 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 945/1850] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:56 2022] PUT /identity/v3/projects/2ddcad57c19049178d00e9aee13a97f6/users/66a6bcc79a3e4b528b524ddb40bc1300/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:56.821427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:56.821729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:56.822015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:56.905577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] MFA Rules not processed for user `66a6bcc79a3e4b528b524ddb40bc1300`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:56.908462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:57.045387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 906/1851] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:57.060230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:57.067956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:57.136743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:57.242546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b9c1-43af-4582-8841-cd4db0c2532b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:57.247071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:57.248360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:57.248847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:57.249371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:57.263404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:57.265173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:57.277897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 946/1852] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:57.448404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:57.449978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:57.493061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:57.506172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb098d6-2de0-4e0b-ab6f-15939e8a83e2 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:57.512587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:57.513414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:57.514067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:57.515715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:57.530277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:57.585181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f0b65-6c47-465d-bf2e-6178109fc6c0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:57.604721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:57.608052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:57.608650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:57.614316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] PATH_INFO: `/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:57.616443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a96ab5e166045a5975c0f2ecb0639bb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:57.619384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:57.674467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:57.676318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c5ab4-eeb2-4984-bb2f-bf9e64cb5699 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:57.697730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 947/1853] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:32:57.847408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a96ab5e166045a5975c0f2ecb0639bb', 'request_id': 'req-70999bd4-2c21-464f-9cf6-80022c84ac0a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:57.869364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a96ab5e166045a5975c0f2ecb0639bb', 'request_id': 'req-70999bd4-2c21-464f-9cf6-80022c84ac0a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:57.891810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a96ab5e166045a5975c0f2ecb0639bb', 'request_id': 'req-70999bd4-2c21-464f-9cf6-80022c84ac0a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:57.942965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] The token cache is being invalidate because project 1a96ab5e166045a5975c0f2ecb0639bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:32:57.944991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:57.953557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-70999bd4-2c21-464f-9cf6-80022c84ac0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:32:57.957211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 907/1854] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:32:57 2022] DELETE /identity/v3/projects/1a96ab5e166045a5975c0f2ecb0639bb => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:57.962302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:57.962894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:57.963346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:57.995703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:57.999024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:58.140144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 948/1855] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:58.148721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.149238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.149584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.176061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:58.177730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:58.191158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 908/1856] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:58.192495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.215248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dff8f-1a26-45b1-88db-9e7fb5b3a307 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.223274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.224053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.224482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.224843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.225551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:32:58.225964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.227659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.243977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 949/1857] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:32:58.248535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.263883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d5d33-8ef8-41a3-adbc-208ca775df8a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.271424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.272545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.272903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.273312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.282859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.282859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.319552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 909/1858] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:58.324269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.342131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e2cad-5653-4e7b-91f0-e8d5a0ffa84f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.385847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:58.408727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.409466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.409751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.410067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.410768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.413144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.441039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 950/1859] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/users => generated 282 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:58.446101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.457957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff377609-3198-46fc-a217-2bb6642050ca admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.470860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.471808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.472094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.472370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.473267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.474752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.489564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 910/1860] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:58.491523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.502201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64126de3-0cfb-494c-9bcc-a50ca060300b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.507578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.508475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.508762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.509038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.509814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.511405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f110f1-a9ad-47ee-863f-e9eb69d84a9d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.524455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 951/1861] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:58.526040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.536636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654ebee-e0aa-4616-8c4b-aa66786cb4d4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.541395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.542290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.542578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.542861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] PATH_INFO: `/v3/projects/7015e42bd7d24ffbb30641471823a988/users/5ba932e4c12c4d129216470b26c1b75a/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.558167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7015e42bd7d24ffbb30641471823a988, user_id=5ba932e4c12c4d129216470b26c1b75a, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.560938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.583457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 911/1862] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:58 2022] PUT /identity/v3/projects/7015e42bd7d24ffbb30641471823a988/users/5ba932e4c12c4d129216470b26c1b75a/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:58.585555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.585845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.586111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.614842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] MFA Rules not processed for user `5ba932e4c12c4d129216470b26c1b75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:58.616101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:58.679674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 952/1863] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:58.684432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.745352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:58.749592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c836d50-f80a-424d-a76b-1b78d4a6ccac admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.756257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.757078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.757587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.758055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.758774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.760662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.785296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 912/1864] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:58.789437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.800420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e0cff-956c-46d9-8e48-039379b0ff2c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.856287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.856893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.857179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.857495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.858186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.860632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.885607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 953/1865] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:32:58 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:32:58.886843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.897095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4ba21-7c39-48c7-b646-679731fc1bb5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.901646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.902305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.902611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.902884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.903603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.905179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.919108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 913/1866] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:58.920493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.930917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219df19-d9b8-4800-b593-354b861c54dc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:58.936073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:58.937034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:58.937384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:58.937667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] PATH_INFO: `/v3/projects/289c3bce6ad34d7b95c9699448e89619/users/cfa51b60b5ef479495a3b826653e1fb5/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:58.952852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=289c3bce6ad34d7b95c9699448e89619, user_id=cfa51b60b5ef479495a3b826653e1fb5, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:58.955514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:58.970153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 954/1867] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:58 2022] PUT /identity/v3/projects/289c3bce6ad34d7b95c9699448e89619/users/cfa51b60b5ef479495a3b826653e1fb5/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:32:58.974400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:58.984537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af17ea-9c7b-43a8-8555-2c5e42d978e5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:59.052070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:59.052763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.053052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.053350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.054081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:59.055668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eadbe5-b475-4101-9f13-4e56e73d0459 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:59.065948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 914/1868] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:32:59.070407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:59.080493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a0f5-5bbd-4d5d-a768-9720b593e66a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:59.085069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:59.085982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.086261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.086535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] PATH_INFO: `/v3/projects/289c3bce6ad34d7b95c9699448e89619/users/cfa51b60b5ef479495a3b826653e1fb5/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.088476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289c3bce6ad34d7b95c9699448e89619, user_id=cfa51b60b5ef479495a3b826653e1fb5, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:59.091123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:59.097840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:59.103250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 955/1869] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:32:59 2022] PUT /identity/v3/projects/289c3bce6ad34d7b95c9699448e89619/users/cfa51b60b5ef479495a3b826653e1fb5/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:32:59.115181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.115475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.115759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.144024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] MFA Rules not processed for user `cfa51b60b5ef479495a3b826653e1fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:32:59.145231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.212776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 915/1870] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:32:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:32:59.224157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:59.225827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.281429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d9135-baf7-496d-b1bb-af9087a226f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:59.284995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:59.285660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.285942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.286218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.295582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:59.297066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:59.306738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 956/1871] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:32:59.386097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:32:59.387596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:32:59.400353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7486b-ff7a-4a72-bd7f-9e9a1f39a074 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:32:59.410420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:32:59.411089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:32:59.411400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:32:59.411680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:32:59.460935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:32:59.471991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:32:59.473507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:32:59.482384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 916/1872] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:01.217554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:01.228153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc50a27-c44b-40d1-9265-ae0c68c3d57a service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:01.236333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:01.237020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:01.237305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:01.237610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:01.239387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:01.305276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:01.306747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618cd02-6103-4a69-b39f-d6d50c05ad2f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:01.317447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 957/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:01.670307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:01.673495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:01.763309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174f0b7-2965-4a48-a553-aba1e1766b54 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:01.768841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:01.769739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:01.770046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:01.770324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] PATH_INFO: `/v3/users/e911c0cc15694e7fad8bc4f5a99d1fd1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:01.771504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=e911c0cc15694e7fad8bc4f5a99d1fd1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:01.773605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:01.825028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e911c0cc15694e7fad8bc4f5a99d1fd1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:01.833043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e911c0cc15694e7fad8bc4f5a99d1fd1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:01.833811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e911c0cc15694e7fad8bc4f5a99d1fd1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:01.847085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e911c0cc15694e7fad8bc4f5a99d1fd1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:01.854337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e911c0cc15694e7fad8bc4f5a99d1fd1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:01.864119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 917/1874] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:01 2022] DELETE /identity/v3/users/e911c0cc15694e7fad8bc4f5a99d1fd1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:01.866490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:01.866858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:01.867463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:01.893724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:01.895113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:01.992563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 958/1875] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:02.003980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.005968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.093798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b41dc-560a-44f1-b6d1-b90b21ed4ace admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.102242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.102642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.103295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.103580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.103708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.103883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.113827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.115261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa83a98-0c6f-469c-ae23-591a1cfb000f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.118263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663599-b381-40cc-8092-c5368fb58c03 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.124036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.125492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.125723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 918/1876] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:02.125842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.126038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] PATH_INFO: `/v3/users/10b84e42d8094b0ba2c18611b35688ad` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.127234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b84e42d8094b0ba2c18611b35688ad)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.129205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.172091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:02.190318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b84e42d8094b0ba2c18611b35688ad'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.212679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b84e42d8094b0ba2c18611b35688ad'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.223050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b84e42d8094b0ba2c18611b35688ad'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.224130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b84e42d8094b0ba2c18611b35688ad'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.233677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b84e42d8094b0ba2c18611b35688ad'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.240759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 959/1877] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:02 2022] DELETE /identity/v3/users/10b84e42d8094b0ba2c18611b35688ad => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:02.245755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.246051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.246347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.268954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:02.270363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.280467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:02.346960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 919/1878] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:02.362633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.364637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.451785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18797-081e-4990-a327-21104fa9c4ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.458377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.459347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.459748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.460445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.472215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.473961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.481425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.482997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.491122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 960/1879] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:02.498542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e354a01-d963-4c5a-91c6-a6cf48c5c091 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.503448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.504221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.504505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.505043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] PATH_INFO: `/v3/projects/edf4b030a96b4180bcd628ebe6aeb966` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.505990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf4b030a96b4180bcd628ebe6aeb966)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.508539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.574828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf4b030a96b4180bcd628ebe6aeb966', 'request_id': 'req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.584372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf4b030a96b4180bcd628ebe6aeb966', 'request_id': 'req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.591403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf4b030a96b4180bcd628ebe6aeb966', 'request_id': 'req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.604564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] The token cache is being invalidate because project edf4b030a96b4180bcd628ebe6aeb966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:02.605018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.606079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.611330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 920/1880] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:02 2022] DELETE /identity/v3/projects/edf4b030a96b4180bcd628ebe6aeb966 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:02.612769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.623775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053ec0-6440-4c82-b212-ba64a7072112 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.724695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.725754 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.726091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.726359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] PATH_INFO: `/v3/users/271a3f57a2f34b9caa623fc64abb5119` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.729036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=271a3f57a2f34b9caa623fc64abb5119)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.732008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:02.799949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271a3f57a2f34b9caa623fc64abb5119'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.813881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271a3f57a2f34b9caa623fc64abb5119'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.816116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:02.831225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271a3f57a2f34b9caa623fc64abb5119'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.832406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271a3f57a2f34b9caa623fc64abb5119'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.841216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271a3f57a2f34b9caa623fc64abb5119'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:02.848654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 961/1881] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:02 2022] DELETE /identity/v3/users/271a3f57a2f34b9caa623fc64abb5119 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:02.870887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:02.875538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:02.899069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834198b-aa6b-473a-814a-2ba3ca0ff2c5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:02.979279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:02.980237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:02.980543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:02.980814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] PATH_INFO: `/v3/projects/9de2c5b2f9a14f1799c00643bc1bdec2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:02.983469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de2c5b2f9a14f1799c00643bc1bdec2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:02.984171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.028589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:03.037685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de2c5b2f9a14f1799c00643bc1bdec2', 'request_id': 'req-d907bc72-3813-4588-8885-0a1841619290'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.047024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de2c5b2f9a14f1799c00643bc1bdec2', 'request_id': 'req-d907bc72-3813-4588-8885-0a1841619290'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.057091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de2c5b2f9a14f1799c00643bc1bdec2', 'request_id': 'req-d907bc72-3813-4588-8885-0a1841619290'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.074534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] The token cache is being invalidate because project 9de2c5b2f9a14f1799c00643bc1bdec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:03.074895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.075630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.077586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 921/1882] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:02 2022] DELETE /identity/v3/projects/9de2c5b2f9a14f1799c00643bc1bdec2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:03.081878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.083698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.097554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41b136-d069-4212-a789-eefce40f2270 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.158641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.159519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.159799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.160074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] PATH_INFO: `/v3/users/d44757e400bd4a1a96341a832c41fddd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.161453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44757e400bd4a1a96341a832c41fddd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.163633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.193667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.195168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.208718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907bc72-3813-4588-8885-0a1841619290 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.213698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.214645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.215022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.215308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] PATH_INFO: `/v3/projects/9c0483f042394228a976f25159820340` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.216690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0483f042394228a976f25159820340)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.218761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.225023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44757e400bd4a1a96341a832c41fddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.234327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44757e400bd4a1a96341a832c41fddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.258538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44757e400bd4a1a96341a832c41fddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.260275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44757e400bd4a1a96341a832c41fddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.272098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63127020-7428-4a3a-82cb-cdebc18e2726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44757e400bd4a1a96341a832c41fddd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.280731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 962/1883] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:03 2022] DELETE /identity/v3/users/d44757e400bd4a1a96341a832c41fddd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:03.288757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0483f042394228a976f25159820340', 'request_id': 'req-3f2e12e7-73d9-4992-b09b-6f08400c0999'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.300160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0483f042394228a976f25159820340', 'request_id': 'req-3f2e12e7-73d9-4992-b09b-6f08400c0999'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.308637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0483f042394228a976f25159820340', 'request_id': 'req-3f2e12e7-73d9-4992-b09b-6f08400c0999'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.324205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] The token cache is being invalidate because project 9c0483f042394228a976f25159820340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:03.324665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.325651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3f2e12e7-73d9-4992-b09b-6f08400c0999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.328324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 922/1884] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:03 2022] DELETE /identity/v3/projects/9c0483f042394228a976f25159820340 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:03.334813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.336005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.336005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.359164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:03.360602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.446771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:03.455842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 963/1885] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:03.465314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.465644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.465941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.487005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:03.488245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.496259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 923/1886] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:03.500542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.510697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc41f67-dda5-47ff-bac3-d8c405494ff9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.515317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.515984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.516269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.516532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.517266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:03.517611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.519221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.533429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 964/1887] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:03.536154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.545572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637407e1-a9ae-4f38-84c6-fcfb21544196 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.550915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.551596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.551883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.552160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.552844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.555035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.584990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 924/1888] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:03.586629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.596335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.598191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff69a7-4fe6-44eb-8843-782057763091 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.598332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.662251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.663059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.663537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.663886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.664657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.666679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.667499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b50a0-1ddf-40d5-89f8-d43a80801955 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.675315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.676924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.677849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.678380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] PATH_INFO: `/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.681290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b91d7e3fcf4ca493ec9224dd3e1b57)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.685031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.709120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 965/1889] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:03.710605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.727271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec125960-4cbf-4797-8258-e8aafab76165 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.727555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:03.735470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.736856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.737449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.737919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.742163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.742163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.771761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b91d7e3fcf4ca493ec9224dd3e1b57', 'request_id': 'req-249c4e71-d854-4da2-88c4-025741f2c894'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.772267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 966/1890] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:03.778450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.783333 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b91d7e3fcf4ca493ec9224dd3e1b57', 'request_id': 'req-249c4e71-d854-4da2-88c4-025741f2c894'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.791952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690657-a215-43fb-b83a-473f13d3124a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.793448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b91d7e3fcf4ca493ec9224dd3e1b57', 'request_id': 'req-249c4e71-d854-4da2-88c4-025741f2c894'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.813945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] The token cache is being invalidate because project 33b91d7e3fcf4ca493ec9224dd3e1b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:03.814632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.828662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-249c4e71-d854-4da2-88c4-025741f2c894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:03.835751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 925/1891] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:03 2022] DELETE /identity/v3/projects/33b91d7e3fcf4ca493ec9224dd3e1b57 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:03.840832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fbbb-0e34-4016-b928-aa5161d870d0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.841127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fbbb-0e34-4016-b928-aa5161d870d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.841455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754fbbb-0e34-4016-b928-aa5161d870d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.862706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7754fbbb-0e34-4016-b928-aa5161d870d0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:03.863917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7754fbbb-0e34-4016-b928-aa5161d870d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.865973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.867028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.867455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.867881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.869082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.872973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 926/1892] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:03.881205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.882970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.883361 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.884022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.895510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 967/1893] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:03.902057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:03.903271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.906003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.907617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:03.911377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 927/1894] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:03.915462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.920136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5daeaac-58aa-4898-b213-1d8863b89c69 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.926697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730aa17-402f-41ee-b309-704d527f6980 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.927674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.928560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.928841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.929112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] PATH_INFO: `/v3/projects/e27c175ef0454bb8906a11314e1e4aa5/users/a15caabce24f44238f04c40900e61db6/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.931535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:03.932209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.932493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.933025 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:03.933792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:03.934122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.935708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.945438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27c175ef0454bb8906a11314e1e4aa5, user_id=a15caabce24f44238f04c40900e61db6, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:03.948151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32fbef3-97da-48c5-82a3-9a840a0b6dbd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:03.951727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 928/1895] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:33:03.952765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:03.962273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 968/1896] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:03 2022] PUT /identity/v3/projects/e27c175ef0454bb8906a11314e1e4aa5/users/a15caabce24f44238f04c40900e61db6/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:03.963081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6593f-7fa1-4e3b-af34-2932e1582ffa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:03.973461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:03.973768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:03.974031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.005292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] MFA Rules not processed for user `a15caabce24f44238f04c40900e61db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:04.006551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.024926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.032030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.032800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.033696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.033998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.034788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.037069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.069517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 929/1897] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:04.071864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.082138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 969/1898] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:04.094322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.095727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.134790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb184059-32ec-4c5f-af4b-3f9350f89751 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.139469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.140684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.140980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.141265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.141999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.144074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.154248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2c87-09a7-479c-83d3-dd1fe80ec5a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.161475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.172325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 930/1899] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:04 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:04.173308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.183902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7b6b4-e88a-4cf4-ac9e-309102f980b8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.189991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.190765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.191188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.191523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.192324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.194060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.206019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 931/1900] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:04.210071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.215045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.216079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.216528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.216957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.224299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f03a2-47a9-4c7b-9174-f5531c2c9458 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.229374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.230116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.230496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.230872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.231695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.233476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.245452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 932/1901] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:04.250360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.264279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae62d3c-5564-4b4f-ad52-e5840fdfbb43 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.269812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.270709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.270996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.271277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] PATH_INFO: `/v3/projects/cefc90d2f25d42ec9f9d5567dda60b58/users/db6294c41e6844faa4279f5fc5c5360d/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.287464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.288001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cefc90d2f25d42ec9f9d5567dda60b58, user_id=db6294c41e6844faa4279f5fc5c5360d, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.289426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.290170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f698b0-507a-4ffe-8c9b-e7cb1237e1c6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.302200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 970/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:04.322050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 933/1903] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:04 2022] PUT /identity/v3/projects/cefc90d2f25d42ec9f9d5567dda60b58/users/db6294c41e6844faa4279f5fc5c5360d/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:04.326263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.327003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.327335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.327667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.338190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48c7-bdf4-4bfc-9ee7-b8d23302899b service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.345491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:04.346694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.365786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.396436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.397406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.397690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.397964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] PATH_INFO: `/v3/users/a15caabce24f44238f04c40900e61db6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.399210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15caabce24f44238f04c40900e61db6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.401228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.408145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 934/1904] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:04.420827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.422542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.457263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15caabce24f44238f04c40900e61db6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:04.466177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15caabce24f44238f04c40900e61db6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:04.480804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15caabce24f44238f04c40900e61db6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:04.482103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15caabce24f44238f04c40900e61db6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:04.485505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9536-ef89-401d-85e6-c5ba97cf0185 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.489551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.490002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-951982a0-3a55-4d9d-ba03-463dd60d39db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15caabce24f44238f04c40900e61db6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:04.490378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.490743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.491120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.493120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.496299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 971/1905] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:04 2022] DELETE /identity/v3/users/a15caabce24f44238f04c40900e61db6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:04.501129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.501551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.501986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.509067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.510555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.521388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 935/1906] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:04.524494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:04.525794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.591347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 972/1907] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:04.597562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:04.599024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:04.610531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b5e29-c681-45cd-bba4-7ed3216428d7 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:04.666525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:04.667205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:04.667472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:04.667747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:04.677957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:04.679382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096f851-5ea8-4c24-9623-cacf903ac4cb service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:04.684757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:04.690507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 936/1908] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:05.175093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.176760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:05.189265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89719511-c9a5-48b5-8807-990cb9c39a72 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.193998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.194745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.195028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.195301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] PATH_INFO: `/v3/projects/e27c175ef0454bb8906a11314e1e4aa5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.197084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27c175ef0454bb8906a11314e1e4aa5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.199230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.258605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:05.262904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27c175ef0454bb8906a11314e1e4aa5', 'request_id': 'req-06a23756-3067-456d-9634-61da620ec5d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:05.272539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27c175ef0454bb8906a11314e1e4aa5', 'request_id': 'req-06a23756-3067-456d-9634-61da620ec5d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:05.278557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27c175ef0454bb8906a11314e1e4aa5', 'request_id': 'req-06a23756-3067-456d-9634-61da620ec5d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:05.290886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] The token cache is being invalidate because project e27c175ef0454bb8906a11314e1e4aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:05.291537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:05.291877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06a23756-3067-456d-9634-61da620ec5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:05.293725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 973/1909] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:05 2022] DELETE /identity/v3/projects/e27c175ef0454bb8906a11314e1e4aa5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:05.298340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56cb4a-d331-498c-83bc-21a2fbf9cf22 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.298800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56cb4a-d331-498c-83bc-21a2fbf9cf22 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.298910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56cb4a-d331-498c-83bc-21a2fbf9cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.318004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ea56cb4a-d331-498c-83bc-21a2fbf9cf22 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:05.319224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ea56cb4a-d331-498c-83bc-21a2fbf9cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:05.381896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.382182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.382869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.410582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 937/1910] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:05.418601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.419004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.419381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.436540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] MFA Rules not processed for user `db6294c41e6844faa4279f5fc5c5360d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:05.437866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:05.450021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:05.451358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:05.462534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 938/1911] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:05.466816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.484695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5940a1-ae68-4682-a29c-a8892e33a2a2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.489203 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.489906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.490188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.490461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.491186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:05.491688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.493942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.514012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 939/1912] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:05.518444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.532846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c7af4-39a3-4470-b441-56013cfbd225 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.539172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.539848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.540108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.540385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.541217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.545458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 974/1913] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:05.545614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.551212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.552723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:05.570680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d06562-6325-4ebf-87d2-16c2a239d2d2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.574918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 940/1914] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:05.576916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.577911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.578267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.578624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.579784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.579923 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.581468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.614992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 975/1915] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:05.619237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.646743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e998f-cff9-47d1-8e12-6e16c48e3b60 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.684204 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84891dea-78e6-416d-bee6-80eb6a68c97b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.692401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.693317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.694091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.694091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.694767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.696791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.708982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.709688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.710043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.710291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.710944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.712912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.721366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 976/1916] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:05.723532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.726432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:05.734873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a73ca3-bf0f-405b-9609-a7f12459ed6f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.739883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 941/1917] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:05.743174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.743509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.744022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.744310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.744588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.745322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.746948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.755136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f55a37-9bdc-4c97-8553-383c3a79d7a5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.761392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 977/1918] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:05.761651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.761651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.761651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.761651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.762873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.763757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.764474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.774660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81841-0bda-4323-9806-5e9b516f729d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.775615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 942/1919] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:05.779315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.782416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.784143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.784690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.785015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] PATH_INFO: `/v3/projects/ace343fcc26740f7a080dec03d9bab44/users/da80b35d27ed4db69b2911af39c29d5e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.791960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39843895-d84e-4144-ab21-d3ffd44c3186 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.796878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.797596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.798321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.798321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.799213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.801015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.813855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace343fcc26740f7a080dec03d9bab44, user_id=da80b35d27ed4db69b2911af39c29d5e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.814058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 943/1920] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:05.817271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.819806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:05.824250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.838383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d411a-6a7b-43f4-b55c-72db0a2283c3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.839594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 978/1921] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:05 2022] PUT /identity/v3/projects/ace343fcc26740f7a080dec03d9bab44/users/da80b35d27ed4db69b2911af39c29d5e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:05.845744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.908797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.909721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.910005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.910354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] PATH_INFO: `/v3/projects/9b78f37984244700a3a5710fe29958ab/users/38f705b79b81492fb52b53449f466dbc/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.922918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23089e-e8de-4e29-aa38-bbe9e280ace8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.929682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:05.930633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.933561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.933829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:05.934607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.935438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b78f37984244700a3a5710fe29958ab, user_id=38f705b79b81492fb52b53449f466dbc, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:05.936222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.938391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d75e3-adb3-49a4-ba98-9db5c3475408 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:05.947889 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 979/1922] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:05.952010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:05.955579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 944/1923] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:05 2022] PUT /identity/v3/projects/9b78f37984244700a3a5710fe29958ab/users/38f705b79b81492fb52b53449f466dbc/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:05.965574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1ed5-81ab-484a-b00f-b58c098ed33a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:05.968061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:05.968940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:05.969289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.002326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] MFA Rules not processed for user `38f705b79b81492fb52b53449f466dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:06.003551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:06.036011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:06.037023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.037406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.037763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] PATH_INFO: `/v3/projects/ace343fcc26740f7a080dec03d9bab44/users/da80b35d27ed4db69b2911af39c29d5e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.039858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace343fcc26740f7a080dec03d9bab44, user_id=da80b35d27ed4db69b2911af39c29d5e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:06.042550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:06.053764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 980/1924] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:05 2022] PUT /identity/v3/projects/ace343fcc26740f7a080dec03d9bab44/users/da80b35d27ed4db69b2911af39c29d5e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:06.083356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 945/1925] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:06.116629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:06.120592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:06.238036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea42e5-e0be-4d33-89bb-4ca8d0c75101 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:06.242833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:06.243769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.244130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.244489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.291738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:06.313078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:06.314940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ff9bb-d94b-4f04-9bd0-cf66b502abfb service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:06.340317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 981/1926] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:06.362169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:06.376397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033d60-2976-435f-99db-472437dc7cb6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:06.443394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:06.459468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:06.460537 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.460900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.461412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] PATH_INFO: `/v3/users/38f705b79b81492fb52b53449f466dbc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.462655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f705b79b81492fb52b53449f466dbc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:06.464971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:06.520430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f705b79b81492fb52b53449f466dbc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:06.529265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f705b79b81492fb52b53449f466dbc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:06.530020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f705b79b81492fb52b53449f466dbc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:06.544818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f705b79b81492fb52b53449f466dbc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:06.554513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f705b79b81492fb52b53449f466dbc'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:06.561128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 946/1927] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:06 2022] DELETE /identity/v3/users/38f705b79b81492fb52b53449f466dbc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:06.566808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e51e-7f8e-4cdf-ad77-17a8facde357 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.566808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e51e-7f8e-4cdf-ad77-17a8facde357 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.566808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e51e-7f8e-4cdf-ad77-17a8facde357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.588304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c511e51e-7f8e-4cdf-ad77-17a8facde357 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:06.589936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c511e51e-7f8e-4cdf-ad77-17a8facde357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:06.662342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 982/1928] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:06.671583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:06.673592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:06.746492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1ff7e-6958-4083-aef2-ccdc4e577ef0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:06.750887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:06.753015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.753315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.753629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.763743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:06.765240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:06.775381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 947/1929] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:06.830167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:06.830547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:06.830892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:06.862556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] MFA Rules not processed for user `da80b35d27ed4db69b2911af39c29d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:06.864114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:06.957291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 983/1930] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:06.975480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:06.977949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.091228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ece318-a5a5-473f-8ae9-abe40a51b301 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.095407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.096147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.096510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.096867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.100091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.115475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.117138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.131534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b8f3b-c325-4f91-a4b0-2de27843262e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.131878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:07.138695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.141217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.141658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.142136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] PATH_INFO: `/v3/projects/9b78f37984244700a3a5710fe29958ab` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.143878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b78f37984244700a3a5710fe29958ab)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.146347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.181073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.182557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449c19-3b7a-41b8-8e07-4c05a08a83fc service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.192852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 948/1931] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:07.198054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b78f37984244700a3a5710fe29958ab', 'request_id': 'req-473f7676-f2be-4993-b351-54537040aab0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:07.206433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b78f37984244700a3a5710fe29958ab', 'request_id': 'req-473f7676-f2be-4993-b351-54537040aab0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:07.208566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:07.213782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b78f37984244700a3a5710fe29958ab', 'request_id': 'req-473f7676-f2be-4993-b351-54537040aab0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:07.229024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] The token cache is being invalidate because project 9b78f37984244700a3a5710fe29958ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:07.229358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:07.230176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-473f7676-f2be-4993-b351-54537040aab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:07.231730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 984/1932] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:07 2022] DELETE /identity/v3/projects/9b78f37984244700a3a5710fe29958ab => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:07.237868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.238122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.238406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.257184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:07.258443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.324874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 949/1933] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:07.332962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.333518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.333608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.354304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:07.355684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.364881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 985/1934] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:07.369303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.379817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7dda-b455-4a58-8568-73f555b028b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.385531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.386227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.386589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.386904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.387634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:07.387963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.389581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.402471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 950/1935] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:07.407117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.417599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009c3d0-7554-490c-9554-af4bce279f77 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.423507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.424268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.424621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.424899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.425653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.427577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.460936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 986/1936] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:07 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:07.465408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.479789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a8b0-dda2-4501-a124-91fdbf51857f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.554875 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.555532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.555832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.556130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.556793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.558993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.582622 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 951/1937] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:07 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:07.587124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.597659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d056a1-65da-48b6-a8fb-cb0dd12e6f4b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.602433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.603102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.603389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.603675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.604425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.606054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.619495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 987/1938] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:07.621402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.632123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e4c24-bd64-452c-8132-792af52efb47 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.636684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.637374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.637660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.637942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.638690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.640337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be9f69-6719-4a99-b98c-818aa3a65cea admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.650091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 952/1939] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:07.654028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.663866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011165-e0fe-4b37-a611-3b59de6c8fe9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.668407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.669280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.669587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.669866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] PATH_INFO: `/v3/projects/553646e8f0884b22ba3e3d5c6b173d1b/users/ab229da11aac429698ad11d65aa58485/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.684735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553646e8f0884b22ba3e3d5c6b173d1b, user_id=ab229da11aac429698ad11d65aa58485, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.688048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.701514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 988/1940] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:07 2022] PUT /identity/v3/projects/553646e8f0884b22ba3e3d5c6b173d1b/users/ab229da11aac429698ad11d65aa58485/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:07.713146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.713472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.713751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.740761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] MFA Rules not processed for user `ab229da11aac429698ad11d65aa58485`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:07.741995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.759076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:07.808835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 953/1941] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:07.820288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:07.822134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:07.882409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684433a8-c71b-412c-969d-80fc95cb0850 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:07.886416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:07.887069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:07.887354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:07.887636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:07.896998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:07.898469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a72c72-cf0e-450f-86bd-0c830965fd95 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:07.908171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 989/1942] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:09.201017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:09.203285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.267304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403e6b-ad2c-464b-98c4-3fee4f5d2bbd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.267854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403e6b-ad2c-464b-98c4-3fee4f5d2bbd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.267854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403e6b-ad2c-464b-98c4-3fee4f5d2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.283996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da713027-0338-4d41-bd8d-9a71722b2165 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:09.286488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef403e6b-ad2c-464b-98c4-3fee4f5d2bbd None None] MFA Rules not processed for user `ab229da11aac429698ad11d65aa58485`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:09.287761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef403e6b-ad2c-464b-98c4-3fee4f5d2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.290061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:09.291068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.291541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.291975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] PATH_INFO: `/v3/users/a0601704fdd847188095e13e2632edf4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.294461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0601704fdd847188095e13e2632edf4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:09.296874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 990/1943] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:09.297156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:09.382522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0601704fdd847188095e13e2632edf4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:09.399233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0601704fdd847188095e13e2632edf4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:09.400047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0601704fdd847188095e13e2632edf4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:09.420185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0601704fdd847188095e13e2632edf4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:09.428882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0601704fdd847188095e13e2632edf4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:09.441146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 954/1944] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:09 2022] DELETE /identity/v3/users/a0601704fdd847188095e13e2632edf4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:09.443468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.444107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.444488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.469442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:09.470848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.547129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 991/1945] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:09.556340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:09.558196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.607377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:09.642795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:09.643569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619bb1-7a09-4c28-97f8-7cf536bb5acd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:09.645995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.647534 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:09.648233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.648598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.648977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.660277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:09.662509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:09.675879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 955/1946] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:09.679872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:09.739380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb56fce-5691-40a3-8e0a-c3d37b0952dc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:09.744461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:09.745316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:09.745848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:09.746285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:09.749541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:09.839895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:09.841773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:09.859109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 992/1947] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:10.284252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:10.285897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:10.300680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3bc03-730b-4dab-ba5c-6d5cb8bee229 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:10.306329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:10.307206 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:10.307495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:10.307778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] PATH_INFO: `/v3/projects/7c91d4aa019a4b19811da935b7347d83` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:10.309233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c91d4aa019a4b19811da935b7347d83)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:10.323218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:10.384390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c91d4aa019a4b19811da935b7347d83', 'request_id': 'req-be7cb589-2360-46c7-9517-a65a8612b52b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.392844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c91d4aa019a4b19811da935b7347d83', 'request_id': 'req-be7cb589-2360-46c7-9517-a65a8612b52b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.399488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c91d4aa019a4b19811da935b7347d83', 'request_id': 'req-be7cb589-2360-46c7-9517-a65a8612b52b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.412556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] The token cache is being invalidate because project 7c91d4aa019a4b19811da935b7347d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:10.412850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.414635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.417393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 956/1948] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:10 2022] DELETE /identity/v3/projects/7c91d4aa019a4b19811da935b7347d83 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:10.420554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:10.431838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccee69-6324-4b3e-beac-2b3d1b8b66b1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:10.491817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:10.492816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:10.493103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:10.493416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] PATH_INFO: `/v3/users/66a6bcc79a3e4b528b524ddb40bc1300` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:10.494618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a6bcc79a3e4b528b524ddb40bc1300)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:10.496526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:10.552146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a6bcc79a3e4b528b524ddb40bc1300'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.560849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a6bcc79a3e4b528b524ddb40bc1300'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.579823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a6bcc79a3e4b528b524ddb40bc1300'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.580978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a6bcc79a3e4b528b524ddb40bc1300'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.582493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:10.591473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0666f7da-90ca-47f5-969f-cd1fc8a4a927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a6bcc79a3e4b528b524ddb40bc1300'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:10.600353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 993/1949] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:10 2022] DELETE /identity/v3/users/66a6bcc79a3e4b528b524ddb40bc1300 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:10.874618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:10.876082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:10.892598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb589-2360-46c7-9517-a65a8612b52b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:10.956315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:10.957197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:10.957509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:10.957804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] PATH_INFO: `/v3/projects/2ddcad57c19049178d00e9aee13a97f6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:10.958985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ddcad57c19049178d00e9aee13a97f6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:10.960916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.007160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ddcad57c19049178d00e9aee13a97f6', 'request_id': 'req-72b56102-1038-48d3-91ab-381f134195a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:11.014593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ddcad57c19049178d00e9aee13a97f6', 'request_id': 'req-72b56102-1038-48d3-91ab-381f134195a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:11.020471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ddcad57c19049178d00e9aee13a97f6', 'request_id': 'req-72b56102-1038-48d3-91ab-381f134195a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:11.021560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:11.034532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] The token cache is being invalidate because project 2ddcad57c19049178d00e9aee13a97f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:11.034819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:11.035447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-72b56102-1038-48d3-91ab-381f134195a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:11.037006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 957/1950] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:10 2022] DELETE /identity/v3/projects/2ddcad57c19049178d00e9aee13a97f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:11.041898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.042211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.042494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.060545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:11.062298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:11.120873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 994/1951] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:11.129239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.129239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.135773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.155353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:11.156642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:11.167798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 958/1952] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:11.175549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.186449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d439750-4d68-4185-af41-b08d3ac420ef None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.191747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.192574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.192866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.193156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.193931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:11.194245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.195998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.214186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 995/1953] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:11.219702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.230204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c5071-443f-4bc3-a881-35143cdf5aa0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.235254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.235935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.236219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.236500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.237208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.239175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.269142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 959/1954] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:11.273186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.283738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee9a39-adb5-4458-a9ad-f8c3f36b3c83 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.353176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.354189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.354756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.355212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.356131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.361400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.384841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 996/1955] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:11.389166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.408223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4952d-90e2-45b1-a818-521790e1767e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.413319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.414091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.414447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.414739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.415475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.417533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.432687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 960/1956] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:11.433874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.444438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6936ea-39a9-4d7b-9a02-f9c9e6459158 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.449195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.449934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.450226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.451017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.451310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.454512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6db75-8720-4f99-9a38-86ecc2c27dad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.467791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 997/1957] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:11.470699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.489216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e198-fb1b-44ed-9443-3d65051c5920 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.502846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.505455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.505455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.505455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] PATH_INFO: `/v3/projects/1da8b76209fb4a05879a97de957ab4c6/users/700de3e0939a4a24a0a011eef90e1f37/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.523076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da8b76209fb4a05879a97de957ab4c6, user_id=700de3e0939a4a24a0a011eef90e1f37, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.525889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.541658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 961/1958] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:11 2022] PUT /identity/v3/projects/1da8b76209fb4a05879a97de957ab4c6/users/700de3e0939a4a24a0a011eef90e1f37/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:11.552287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.552577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.552858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.580133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] MFA Rules not processed for user `700de3e0939a4a24a0a011eef90e1f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:11.581296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:11.630594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:11.651862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 998/1959] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:11.656505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.723077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a07a5-d7cb-4459-b01f-a3469974759c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.727675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.728330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.728617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.728898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.729635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.731526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.762414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 962/1960] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:11.766610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.782056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cae84a-c9e5-4aa9-a8ae-8f376acea551 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.842629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.843888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.844256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.844569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.845326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.847279 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.868748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 999/1961] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:11 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:11.872666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.883453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8600529-22fc-49d4-97e7-31924e4f48dc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.888081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.888728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.889009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.889285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.890616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.892823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.903603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 963/1962] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:11.907742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.917696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6273d-70cb-45f2-ad9d-ed139df3f83a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:11.922493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:11.923368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:11.923652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:11.923929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] PATH_INFO: `/v3/projects/e5c2a8b2995b409faac1f77655348c03/users/c4c38f50aaff4e6aa0de2a4a745d5997/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:11.940974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c2a8b2995b409faac1f77655348c03, user_id=c4c38f50aaff4e6aa0de2a4a745d5997, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:11.943647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:11.957010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1000/1963] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:11 2022] PUT /identity/v3/projects/e5c2a8b2995b409faac1f77655348c03/users/c4c38f50aaff4e6aa0de2a4a745d5997/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:11.960926 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:11.971193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478365c6-5b89-4771-90d0-2af8d9da2c2e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:12.029633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:12.030301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.030608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.030883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.031597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:12.033166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d141f0-bfb2-4dd0-aa07-c9edf06cf45f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:12.038145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:12.043681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 964/1964] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:12.048136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:12.058520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b75b-07ba-40c0-aea8-f259ec0ecb48 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:12.063713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:12.064596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.064872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.065141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] PATH_INFO: `/v3/projects/e5c2a8b2995b409faac1f77655348c03/users/c4c38f50aaff4e6aa0de2a4a745d5997/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.067169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c2a8b2995b409faac1f77655348c03, user_id=c4c38f50aaff4e6aa0de2a4a745d5997, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:12.069869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:12.081002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1001/1965] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:12 2022] PUT /identity/v3/projects/e5c2a8b2995b409faac1f77655348c03/users/c4c38f50aaff4e6aa0de2a4a745d5997/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:12.091751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.092043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.092326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.118639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] MFA Rules not processed for user `c4c38f50aaff4e6aa0de2a4a745d5997`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:12.119782 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:12.189500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 965/1966] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:12.200249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:12.202189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:12.260201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900a87d-6e79-4040-a5fd-e862301d1558 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:12.263831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:12.264478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:12.264764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:12.265684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:12.274567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:12.276033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb91c5-af75-4e3a-ae9d-a0efde1ab10d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:12.286038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1002/1967] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:13.072349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:13.170996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8166-7d8b-4c53-bf17-e9f905ff8dee None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:13.176416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:13.178179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:13.178642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:13.178943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] PATH_INFO: `/v3/users/700de3e0939a4a24a0a011eef90e1f37` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:13.180387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700de3e0939a4a24a0a011eef90e1f37)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:13.183145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:13.260777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700de3e0939a4a24a0a011eef90e1f37'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:13.273469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700de3e0939a4a24a0a011eef90e1f37'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:13.274221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700de3e0939a4a24a0a011eef90e1f37'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:13.289122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700de3e0939a4a24a0a011eef90e1f37'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:13.308325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700de3e0939a4a24a0a011eef90e1f37'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:13.326355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 966/1968] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:13 2022] DELETE /identity/v3/users/700de3e0939a4a24a0a011eef90e1f37 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:13.328558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:13.329050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:13.329310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:13.349074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:13.350463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:13.387939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:13.427371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1003/1969] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:13.447156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:13.448839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:13.456940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:13.533371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac6c8-b0ca-4730-8f9f-942547734988 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:13.537795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:13.538459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:13.538748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:13.539027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:13.549323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:13.550798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:13.563106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 967/1970] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:14.048517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:14.050354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:14.075673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce48139-50cb-41db-ab3a-61c8f1a42573 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:14.093171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:14.094085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.094368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.094643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] PATH_INFO: `/v3/projects/1da8b76209fb4a05879a97de957ab4c6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.095917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da8b76209fb4a05879a97de957ab4c6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:14.097764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:14.150462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da8b76209fb4a05879a97de957ab4c6', 'request_id': 'req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.165174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da8b76209fb4a05879a97de957ab4c6', 'request_id': 'req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.181960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da8b76209fb4a05879a97de957ab4c6', 'request_id': 'req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.203767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] The token cache is being invalidate because project 1da8b76209fb4a05879a97de957ab4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:14.204094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.204993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.206903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1004/1971] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:14 2022] DELETE /identity/v3/projects/1da8b76209fb4a05879a97de957ab4c6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:14.210942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:14.222366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6037468-517d-4eff-a0c1-741cf00cf6bd service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:14.300994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:14.302349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.302800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.303176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] PATH_INFO: `/v3/users/c4c38f50aaff4e6aa0de2a4a745d5997` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.304723 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c38f50aaff4e6aa0de2a4a745d5997)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:14.307704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:14.386566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c38f50aaff4e6aa0de2a4a745d5997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.397170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c38f50aaff4e6aa0de2a4a745d5997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.398356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c38f50aaff4e6aa0de2a4a745d5997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.430949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c38f50aaff4e6aa0de2a4a745d5997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.443648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc5d19ca-1bdc-4f5d-add8-162b062c00b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c38f50aaff4e6aa0de2a4a745d5997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.456449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 968/1972] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:14 2022] DELETE /identity/v3/users/c4c38f50aaff4e6aa0de2a4a745d5997 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:14.721628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:14.723497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:14.739031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c2b4f-2028-4d3d-a57b-86ddd617e9c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:14.802213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:14.803177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.803454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.803769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] PATH_INFO: `/v3/projects/e5c2a8b2995b409faac1f77655348c03` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.805255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c2a8b2995b409faac1f77655348c03)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:14.807262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:14.866405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c2a8b2995b409faac1f77655348c03', 'request_id': 'req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.882326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c2a8b2995b409faac1f77655348c03', 'request_id': 'req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.891721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c2a8b2995b409faac1f77655348c03', 'request_id': 'req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.895299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:14.914443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] The token cache is being invalidate because project e5c2a8b2995b409faac1f77655348c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:14.914824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.915605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aca5fcc-9bc3-405e-a9c6-2ee4d5165911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:14.917700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1005/1973] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:14 2022] DELETE /identity/v3/projects/e5c2a8b2995b409faac1f77655348c03 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:14.923573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:14.923573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:14.923888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:14.948532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:14.949840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:14.995768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:15.044073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 969/1974] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:15.051668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.051978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.052243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.071998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:15.073201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.081886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1006/1975] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:15.087685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.098643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66eb41-6144-46a1-a9f9-3a70c59f4dd4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.105937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.106626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.106896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.107191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.107920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:15.108224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.110389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.130226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.136422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 970/1976] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:15.142703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7627e6a-04e7-4b75-8f13-143549a86c04 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.150047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.154308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.154858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.155478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.156341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.158454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.220350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1007/1977] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:15 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:15.224575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.236113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda72435-18db-4941-90c7-275b016e4c4e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.314402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.315183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.315539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.315914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.316659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.318855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.341560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 971/1978] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:15 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:15.346021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.357190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068f180-3f4f-4c02-9f95-ded4f84b05f5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.363021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.363849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.364143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.364483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.366131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.367915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.384017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1008/1979] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:15.384979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.402775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0bbc7-9756-4745-9a93-3b73798d918f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.416621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.418046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.418046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.418046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.419327 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.424710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a155-782c-4647-b313-bd54e7a02bc6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.444156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 972/1980] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:15.445429 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.456142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f9c9d-8c98-463e-9116-21d41d217564 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.461433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.462631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.463050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.463433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] PATH_INFO: `/v3/projects/760f4fba340340b48eb6d85e116d05bc/users/be9cf3b3d4944a838744c0ec0b4e520b/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.481862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760f4fba340340b48eb6d85e116d05bc, user_id=be9cf3b3d4944a838744c0ec0b4e520b, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.484793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.504057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1009/1981] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:15 2022] PUT /identity/v3/projects/760f4fba340340b48eb6d85e116d05bc/users/be9cf3b3d4944a838744c0ec0b4e520b/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:15.514654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.515043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.515401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.565129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] MFA Rules not processed for user `be9cf3b3d4944a838744c0ec0b4e520b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:15.567521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.703712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 973/1982] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:15.716083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.719916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.750927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:15.753503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.791809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:15.803123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608867-be93-4276-8133-7733bd949ac6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.807642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.808800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.809097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.809564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.822314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.823980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230357f-2a0e-44dd-bde5-9c8470c0c05b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.837911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1010/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:15.839611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b60783-0b01-4d2d-8144-525ec9f4891c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.839909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b60783-0b01-4d2d-8144-525ec9f4891c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.840359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b60783-0b01-4d2d-8144-525ec9f4891c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.842768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1011/1984] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:33:15.850509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac74c5-3f9e-4713-94b0-3c9bb1503cbc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:15.857646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.858176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.859201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.862655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:15.865487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:15.868224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:15.870746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] PATH_INFO: `/v3/users/5ba932e4c12c4d129216470b26c1b75a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:15.873512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba932e4c12c4d129216470b26c1b75a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:15.879962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:15.893383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:15.896663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:15.963119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba932e4c12c4d129216470b26c1b75a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:15.977098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba932e4c12c4d129216470b26c1b75a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:15.978297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba932e4c12c4d129216470b26c1b75a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:16.009193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba932e4c12c4d129216470b26c1b75a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:16.028883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7451b304-7de6-44c7-b5ff-ffbcbf484d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba932e4c12c4d129216470b26c1b75a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:16.043712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 974/1985] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:15 2022] DELETE /identity/v3/users/5ba932e4c12c4d129216470b26c1b75a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:16.049804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:16.050165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:16.050365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:16.054375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1012/1986] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:33:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:16.088117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:16.089606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:16.090794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:16.091639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:16.125996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cae3e5-baa6-44a9-8cb0-b117fb01a90d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:16.167636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:16.256987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 975/1987] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:16.266673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:16.268949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:16.307757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:16.308701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:16.309255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:16.309575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:16.321226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:16.323100 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:16.335983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1013/1988] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:16.370807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:16.402510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3235e-29b6-4018-b328-fd8e3c6e0ea2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:16.430752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:16.431437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:16.431887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:16.432177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:16.455523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:16.457139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:16.469773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 976/1989] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:16.975093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:16.976522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:16.990540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ae99-bf93-4d13-98c1-f2734f67dbca service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:16.995127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:16.995992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:16.996275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:16.996555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] PATH_INFO: `/v3/projects/7015e42bd7d24ffbb30641471823a988` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:16.997769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7015e42bd7d24ffbb30641471823a988)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:16.999756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:17.046002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7015e42bd7d24ffbb30641471823a988', 'request_id': 'req-01247772-aa74-464f-91f0-c4c1b830db95'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.052915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7015e42bd7d24ffbb30641471823a988', 'request_id': 'req-01247772-aa74-464f-91f0-c4c1b830db95'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.058764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7015e42bd7d24ffbb30641471823a988', 'request_id': 'req-01247772-aa74-464f-91f0-c4c1b830db95'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.070001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] The token cache is being invalidate because project 7015e42bd7d24ffbb30641471823a988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:17.070295 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.070929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.072439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1014/1990] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:16 2022] DELETE /identity/v3/projects/7015e42bd7d24ffbb30641471823a988 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:17.076530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:17.086821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd103-53c5-4d97-b430-97c61edd2a12 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:17.143494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:17.144362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.144649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.144931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] PATH_INFO: `/v3/users/cfa51b60b5ef479495a3b826653e1fb5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.146152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa51b60b5ef479495a3b826653e1fb5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:17.148060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:17.194630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa51b60b5ef479495a3b826653e1fb5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.201736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa51b60b5ef479495a3b826653e1fb5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.202907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa51b60b5ef479495a3b826653e1fb5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.214911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa51b60b5ef479495a3b826653e1fb5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.222394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f3180aa-569e-40dd-b733-6b1101b2eae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa51b60b5ef479495a3b826653e1fb5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.228355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 977/1991] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:17 2022] DELETE /identity/v3/users/cfa51b60b5ef479495a3b826653e1fb5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:17.513560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:17.514992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:17.529394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01247772-aa74-464f-91f0-c4c1b830db95 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:17.586270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:17.587144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.587428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.587707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] PATH_INFO: `/v3/projects/289c3bce6ad34d7b95c9699448e89619` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.588890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=289c3bce6ad34d7b95c9699448e89619)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:17.590825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:17.627671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:17.638462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289c3bce6ad34d7b95c9699448e89619', 'request_id': 'req-3583cebc-5d03-4805-a02b-b5d345b754b5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.645640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289c3bce6ad34d7b95c9699448e89619', 'request_id': 'req-3583cebc-5d03-4805-a02b-b5d345b754b5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.651092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289c3bce6ad34d7b95c9699448e89619', 'request_id': 'req-3583cebc-5d03-4805-a02b-b5d345b754b5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.662168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] The token cache is being invalidate because project 289c3bce6ad34d7b95c9699448e89619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:17.662458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.663110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3583cebc-5d03-4805-a02b-b5d345b754b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:17.664726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1015/1992] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:17 2022] DELETE /identity/v3/projects/289c3bce6ad34d7b95c9699448e89619 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:17.669698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.670121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.670307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.689398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:17.690607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:17.758257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 978/1993] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:17.765960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.766240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.766512 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.788864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:17.790309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:17.800501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1016/1994] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:17.805078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:17.812641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:17.817083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965e7e5-f76f-4ec1-a002-427243eb1fbf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:17.826880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:17.827577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.828262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.829153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.831193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:17.831193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:17.832317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:17.846247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 979/1995] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:17.853207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:17.863283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8775-337c-4bb5-9bb3-082d5a101ba7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:17.868347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:17.869039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:17.869323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:17.869650 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:17.871008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:17.872379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:17.908090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1017/1996] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:17 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:17.910009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:17.922837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c4e10-0d60-41ce-bf43-07b1cf3a5bf9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.014662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.015629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.015915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.016199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.016908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.018944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.041392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 980/1997] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:17 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:18.045407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.056532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf07a-b6f3-44ec-a2cf-2950a5b6ddab admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.062060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.062940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.063319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.063692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.064523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.066434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.079383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1018/1998] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:18.084376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.094297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d265-a96d-47a8-8b56-141b6e189df5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.099399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.100169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.100540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.100903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.101858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.104095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9262-bf86-49ee-a614-1729903d6932 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.116020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 981/1999] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:18.120257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.131016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d261f05-1046-4530-bb60-a0481667edec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.135852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.136796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.137154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.137533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] PATH_INFO: `/v3/projects/37753d9685f24be5968c4002cf31a1d0/users/88deb209546444308278f579bfc86782/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.151684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37753d9685f24be5968c4002cf31a1d0, user_id=88deb209546444308278f579bfc86782, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.154364 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.166868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1019/2000] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:18 2022] PUT /identity/v3/projects/37753d9685f24be5968c4002cf31a1d0/users/88deb209546444308278f579bfc86782/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:18.178362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.178732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.179106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.212532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] MFA Rules not processed for user `88deb209546444308278f579bfc86782`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:18.215304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:18.314829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 982/2001] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:18.319625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.393234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f720-2c05-48cd-9170-09fd02adf656 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.400322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.400871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.401265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.401580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.402281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.404193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.441156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1020/2002] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:18 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:18.445286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.457951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd6f33-9bff-42ab-8dec-08454c35b7a8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.547515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.548201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.548481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.548753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.549474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.551377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.559581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:18.579621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 983/2003] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:18 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:18.583515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.596501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5c39b-ac00-415b-b71f-6f2d34233095 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.601813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.602491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.602775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.603582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.603863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.605658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.617985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1021/2004] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:18.622202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.633381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd8c7-b98d-458c-9bfb-eb038e5bdaec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.638903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.639767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.640544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.640821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] PATH_INFO: `/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b/users/18e0b492d7cc4d24915c23301beb614c/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.657606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4947dc5f91475ca3cec45c0b8d730b, user_id=18e0b492d7cc4d24915c23301beb614c, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.660402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.675247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 984/2005] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:18 2022] PUT /identity/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b/users/18e0b492d7cc4d24915c23301beb614c/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:18.678997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.691534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c184d9f-c2bd-409a-ad0b-928e3b56f492 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.729581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:18.767876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.768589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.768871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.769161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.769941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.771851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b9c9e-d1fe-463d-b51d-95f1ad7af7a9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.784997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1022/2006] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:18.789457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:18.801693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077572-07ab-4e6a-92e9-e4b877b0b073 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:18.806414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:18.807303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.807692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.808231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] PATH_INFO: `/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b/users/18e0b492d7cc4d24915c23301beb614c/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.810312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4947dc5f91475ca3cec45c0b8d730b, user_id=18e0b492d7cc4d24915c23301beb614c, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:18.813585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:18.824571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 985/2007] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:18 2022] PUT /identity/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b/users/18e0b492d7cc4d24915c23301beb614c/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:18.835281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:18.835669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:18.835955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:18.866529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] MFA Rules not processed for user `18e0b492d7cc4d24915c23301beb614c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:18.868274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:19.025119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1023/2008] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:19.043416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:19.045559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:19.212750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d153d-a197-4fd8-a51d-fc954b315471 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:19.224492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:19.225749 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:19.227113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:19.228168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:19.257186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:19.259277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66af6b6-b05c-4e22-a555-ef9cad8febd4 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:19.288821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 986/2009] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:20.876477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:20.878018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:20.983115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8f8d-90bb-4a3a-afc8-0bba9ff2149c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:20.994475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:20.995395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:20.995783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:20.996069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] PATH_INFO: `/v3/users/be9cf3b3d4944a838744c0ec0b4e520b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:20.997869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9cf3b3d4944a838744c0ec0b4e520b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:21.001071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:21.053098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9cf3b3d4944a838744c0ec0b4e520b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.065509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9cf3b3d4944a838744c0ec0b4e520b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.069708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:21.091771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9cf3b3d4944a838744c0ec0b4e520b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.093483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9cf3b3d4944a838744c0ec0b4e520b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.104268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9cf3b3d4944a838744c0ec0b4e520b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.112493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1024/2010] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:20 2022] DELETE /identity/v3/users/be9cf3b3d4944a838744c0ec0b4e520b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:21.117073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.117611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.117891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.137135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:21.138488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.176809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:21.178598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.217476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 987/2011] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:21.228004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:21.229514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.260792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca1d3-54d1-4e3e-a2a1-db113d3c792a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:21.266262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:21.267142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.267416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.267692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] PATH_INFO: `/v3/users/88deb209546444308278f579bfc86782` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.268921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88deb209546444308278f579bfc86782)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:21.270905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:21.295095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bd6af-f2f5-41fd-b268-958124712b40 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:21.299685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:21.300367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.301291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.302203 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.313082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:21.314553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e8ffc-ed41-485e-82bf-21092c4e6cac service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:21.318642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:21.320424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88deb209546444308278f579bfc86782'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.326505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 988/2012] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:21.333904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88deb209546444308278f579bfc86782'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.347405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88deb209546444308278f579bfc86782'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.348140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88deb209546444308278f579bfc86782'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.355254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88deb209546444308278f579bfc86782'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.361913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1025/2013] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:21 2022] DELETE /identity/v3/users/88deb209546444308278f579bfc86782 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:21.366943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.368726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.368998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.388490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:21.390374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.472766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 989/2014] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:21.487943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:21.490839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.560051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950040e0-2bff-464e-b52b-c74432a2280c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:21.564158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:21.565783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.565783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.565783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.573137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:21.580885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:21.588838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316b5e6-fb2b-420f-aff5-c4dac0c33a42 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:21.618545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1026/2015] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:21.691928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:21.693472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.705673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec813a-c0ab-416b-9e78-8dec41a5ef94 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:21.710758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:21.711734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.711954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.712233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] PATH_INFO: `/v3/projects/760f4fba340340b48eb6d85e116d05bc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.713666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=760f4fba340340b48eb6d85e116d05bc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:21.715567 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:21.770755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760f4fba340340b48eb6d85e116d05bc', 'request_id': 'req-69665cef-20c3-4d16-b578-e3f2ded739f3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.781458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760f4fba340340b48eb6d85e116d05bc', 'request_id': 'req-69665cef-20c3-4d16-b578-e3f2ded739f3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.789152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760f4fba340340b48eb6d85e116d05bc', 'request_id': 'req-69665cef-20c3-4d16-b578-e3f2ded739f3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.803693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] The token cache is being invalidate because project 760f4fba340340b48eb6d85e116d05bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:21.803989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.805124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-69665cef-20c3-4d16-b578-e3f2ded739f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:21.807106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 990/2016] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:21 2022] DELETE /identity/v3/projects/760f4fba340340b48eb6d85e116d05bc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:21.813037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.814417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.814417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.837372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:21.838877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.923729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1027/2017] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:21.932332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:21.932853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:21.933606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:21.968051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:21.969283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:21.978667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 991/2018] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:21.983238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:21.995167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bc1f4-0e2b-4a58-ab36-e32d1ee2c7b5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.001909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.003056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.003482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.003900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.004774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:22.005226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.007868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.021823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1028/2019] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:22.040626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.050023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4d89d-edd8-4671-8839-8cb177bd7961 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.055189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.055843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.056123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.056398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.057440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.060226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.072591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.074439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.088494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44560666-80d7-467a-9027-4c1174a40278 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.095161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.096307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.096840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.097360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] PATH_INFO: `/v3/projects/37753d9685f24be5968c4002cf31a1d0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.099158 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37753d9685f24be5968c4002cf31a1d0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.102031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.105028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 992/2020] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:22 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:22.105933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.190079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37753d9685f24be5968c4002cf31a1d0', 'request_id': 'req-77054b5f-84e4-4698-897b-57e2709ffb31'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.197845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:22.200555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37753d9685f24be5968c4002cf31a1d0', 'request_id': 'req-77054b5f-84e4-4698-897b-57e2709ffb31'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.216036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37753d9685f24be5968c4002cf31a1d0', 'request_id': 'req-77054b5f-84e4-4698-897b-57e2709ffb31'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.228612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237265f7-3bd2-4cf2-aaac-e6445ffe2f97 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.233508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] The token cache is being invalidate because project 37753d9685f24be5968c4002cf31a1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:22.234194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.234705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.237046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1029/2021] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:22 2022] DELETE /identity/v3/projects/37753d9685f24be5968c4002cf31a1d0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:22.241271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.244093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.312279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.312959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.313243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.313552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.314242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.316320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.344193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 993/2022] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:22 2022] POST /identity/v3/users => generated 283 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:22.348388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.349224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77054b5f-84e4-4698-897b-57e2709ffb31 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.350460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.357985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.360485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.360897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.361930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] PATH_INFO: `/v3/users/18e0b492d7cc4d24915c23301beb614c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.363270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e0b492d7cc4d24915c23301beb614c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.364867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1ba07-b1b2-4170-9086-c8ea9a5d1910 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.368724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.370570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.371662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.372249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.372562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.374161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.376740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.380539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:22.389020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 994/2023] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:22.392841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.413187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7be11-d90b-4c3a-ae72-1943410ff4ac admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.424930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.426028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.429394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.429660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.430454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.432754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.451773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 995/2024] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:22.452967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.458420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e0b492d7cc4d24915c23301beb614c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.465585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543eb129-eb5e-4635-9bff-35ce24806deb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.471983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.472883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.473166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.473278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e0b492d7cc4d24915c23301beb614c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.473521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] PATH_INFO: `/v3/projects/5eec78cc6e6f43708497e41c53f54fc5/users/777c2f7283914f638dcf6e503ddfd36b/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.497731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e0b492d7cc4d24915c23301beb614c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.497731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e0b492d7cc4d24915c23301beb614c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.497914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eec78cc6e6f43708497e41c53f54fc5, user_id=777c2f7283914f638dcf6e503ddfd36b, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.497914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.498669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b42d8206-5447-4c58-b9bb-6c8ce373cc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e0b492d7cc4d24915c23301beb614c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:22.508647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1030/2025] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:22 2022] DELETE /identity/v3/users/18e0b492d7cc4d24915c23301beb614c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:22.510081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 996/2026] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:22 2022] PUT /identity/v3/projects/5eec78cc6e6f43708497e41c53f54fc5/users/777c2f7283914f638dcf6e503ddfd36b/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:22.521281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.522099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.522413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.560645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] MFA Rules not processed for user `777c2f7283914f638dcf6e503ddfd36b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:22.562418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.641725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1031/2027] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:22.650245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.652141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.722963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9cf88-a1ba-45dd-a496-7a3c70fe9bce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.779686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.780483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.780774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.781076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.794114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.795676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44094ce-2b2c-43c3-a0b7-d95222bf70e6 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.803986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:22.805908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:22.811267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 997/2028] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:22.915164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22abc7-dfcb-4c08-a98b-a61431b1d9df None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:22.924402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:22.925589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:22.925934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:22.929739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] PATH_INFO: `/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:22.931517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4947dc5f91475ca3cec45c0b8d730b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:22.935120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:22.959063 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:23.019372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4947dc5f91475ca3cec45c0b8d730b', 'request_id': 'req-c60717fa-c656-46fe-a598-2b52841d2cdc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:23.028162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4947dc5f91475ca3cec45c0b8d730b', 'request_id': 'req-c60717fa-c656-46fe-a598-2b52841d2cdc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:23.037831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4947dc5f91475ca3cec45c0b8d730b', 'request_id': 'req-c60717fa-c656-46fe-a598-2b52841d2cdc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:23.051715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] The token cache is being invalidate because project 8d4947dc5f91475ca3cec45c0b8d730b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:23.052005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:23.052680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c60717fa-c656-46fe-a598-2b52841d2cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:23.054607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1032/2029] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:22 2022] DELETE /identity/v3/projects/8d4947dc5f91475ca3cec45c0b8d730b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:23.059579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.059873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.060155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.066679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:23.080869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:23.082252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:23.159757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 998/2030] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:23.172871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.173161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.173473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.198563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:23.199801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:23.208941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1033/2031] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:23.214819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.226942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d7ff2-9ce0-4bdc-8d42-944575c6854f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.234135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.235066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.235352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.235634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.236447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:23.236863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.238557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.252720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 999/2032] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:23.257021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.278829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774e2a1-61f1-4b10-91cf-d2de7746c563 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.286937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.287622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.287908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.288188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.288892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.290919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.322384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1034/2033] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:23.326974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.338884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7813987-04b0-4c8b-87a4-7f1ceeedc82f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.400930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.401734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.402200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.402592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.408912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.408912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.435011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1000/2034] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:23.439338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.454905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285a32-5ae3-4b1d-bdb2-42432b9803ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.455681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.456620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.456898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.457170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.457959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.459573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.470762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1035/2035] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:23.474775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.485466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e72c-ed20-4bbf-99dc-8d6ebd199133 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.490445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.491256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.491539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.491813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.492543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.494167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00299a71-85d0-4013-bc3c-168f633161f9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.505073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1001/2036] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:23.508973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.520381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe95c-8318-42d3-a2ec-7ceb8e061852 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.526049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.526950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.527233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.527507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] PATH_INFO: `/v3/projects/8a4f88739c224223a4dc2b522b22ac69/users/f610510df1fa4595be2f2789c08d59f4/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.545085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4f88739c224223a4dc2b522b22ac69, user_id=f610510df1fa4595be2f2789c08d59f4, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.548117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.565557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1036/2037] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:23 2022] PUT /identity/v3/projects/8a4f88739c224223a4dc2b522b22ac69/users/f610510df1fa4595be2f2789c08d59f4/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:23.576397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.576758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.577662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.605021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] MFA Rules not processed for user `f610510df1fa4595be2f2789c08d59f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:23.606282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:23.666742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:23.672579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1002/2038] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:23.676847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.741255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b40b1-db3b-40ea-bc3c-dede79c35f1b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.746885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.747583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.747882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.748149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.749402 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.751387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.783445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1037/2039] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:23.784843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.796655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31ed02-f0b2-4167-92b2-73282a2b1879 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.858727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.859425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.859712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.859995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.860700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.862825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.887125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1003/2040] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:23 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:23.888333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.899004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b418-1955-4640-8259-70d7d6d0f04b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.903647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.904311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.904596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.904878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.906164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.907799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.922014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1038/2041] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:23.923427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.935020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38c6b2-4101-4edf-a206-49660bbbddc8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:23.940090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:23.941138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:23.941563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:23.941851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] PATH_INFO: `/v3/projects/9623b56d566e4b49ac3def9cdd0d1765/users/a578185e7dcb4cf6aec3dcadd4808b92/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:23.958777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9623b56d566e4b49ac3def9cdd0d1765, user_id=a578185e7dcb4cf6aec3dcadd4808b92, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:23.961899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:23.977974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1004/2042] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:23 2022] PUT /identity/v3/projects/9623b56d566e4b49ac3def9cdd0d1765/users/a578185e7dcb4cf6aec3dcadd4808b92/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:23.981878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:23.992556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a716-f9e4-4802-93ff-d229b53ab663 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:24.013145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:24.067174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:24.067863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.068152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.068433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.069164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:24.070832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae5583-d7d7-45c8-9fda-611361b57c74 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:24.081793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1039/2043] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:24.085976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:24.096588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea47a0-0f28-4a26-8417-e2f12d7a5068 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:24.101505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:24.102392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.102678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.102984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] PATH_INFO: `/v3/projects/9623b56d566e4b49ac3def9cdd0d1765/users/a578185e7dcb4cf6aec3dcadd4808b92/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.104956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9623b56d566e4b49ac3def9cdd0d1765, user_id=a578185e7dcb4cf6aec3dcadd4808b92, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:24.107708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:24.119939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1005/2044] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:24 2022] PUT /identity/v3/projects/9623b56d566e4b49ac3def9cdd0d1765/users/a578185e7dcb4cf6aec3dcadd4808b92/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:24.130942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.131344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.131696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.158408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] MFA Rules not processed for user `a578185e7dcb4cf6aec3dcadd4808b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:24.159631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:24.245692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1040/2045] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:24.254463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:24.256335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:24.323182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4fb2-9173-4fcf-88c0-2b7d6ad4120a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:24.327059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:24.327713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.328054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.328331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.338100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:24.339529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587e263-3e30-4eab-9142-3a28263d4dc6 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:24.349484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1006/2046] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:24.427700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:24.522780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a73ea-30d8-499e-8351-2d13938c8595 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:24.528305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:24.529250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.529498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.529779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] PATH_INFO: `/v3/users/f610510df1fa4595be2f2789c08d59f4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.531536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f610510df1fa4595be2f2789c08d59f4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:24.533537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:24.614934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:24.622053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f610510df1fa4595be2f2789c08d59f4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:24.631412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f610510df1fa4595be2f2789c08d59f4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:24.645014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f610510df1fa4595be2f2789c08d59f4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:24.645719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f610510df1fa4595be2f2789c08d59f4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:24.652872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f610510df1fa4595be2f2789c08d59f4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:24.658851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1041/2047] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:24 2022] DELETE /identity/v3/users/f610510df1fa4595be2f2789c08d59f4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:24.663388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.663705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.663980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.681307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:24.682456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:24.730197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:24.743544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1007/2048] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:24.753917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:24.755336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:24.811728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8de32c-4be4-4b48-82a1-77e0b1624e77 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:24.815382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:24.816033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:24.816315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:24.816680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:24.825943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:24.827921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:24.837725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1042/2049] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:25.110824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:25.112474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:25.125558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3621744-cb56-4698-92ca-de6964d7b65d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:25.131117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:25.132236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:25.132637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:25.133052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] PATH_INFO: `/v3/projects/8a4f88739c224223a4dc2b522b22ac69` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:25.134370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4f88739c224223a4dc2b522b22ac69)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:25.136417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:25.196714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4f88739c224223a4dc2b522b22ac69', 'request_id': 'req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.206406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4f88739c224223a4dc2b522b22ac69', 'request_id': 'req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.214249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4f88739c224223a4dc2b522b22ac69', 'request_id': 'req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.229719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] The token cache is being invalidate because project 8a4f88739c224223a4dc2b522b22ac69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:25.230007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.230672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.233882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1008/2050] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:25 2022] DELETE /identity/v3/projects/8a4f88739c224223a4dc2b522b22ac69 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:25.237741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:25.248059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf50552-f667-496c-bc29-9f4e9f4badd4 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:25.332227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:25.333129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:25.333510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:25.333722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] PATH_INFO: `/v3/users/a578185e7dcb4cf6aec3dcadd4808b92` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:25.336017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a578185e7dcb4cf6aec3dcadd4808b92)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:25.336919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:25.401705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a578185e7dcb4cf6aec3dcadd4808b92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.416525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a578185e7dcb4cf6aec3dcadd4808b92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.426823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:25.436238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a578185e7dcb4cf6aec3dcadd4808b92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.437831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a578185e7dcb4cf6aec3dcadd4808b92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.450253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00f2bd7a-7f23-467e-bdda-fbae2d630a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a578185e7dcb4cf6aec3dcadd4808b92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.459483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1043/2051] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:25 2022] DELETE /identity/v3/users/a578185e7dcb4cf6aec3dcadd4808b92 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:25.779601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:25.782363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:25.800077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964b1d-1941-4a10-aa67-d2a1e6bc4b35 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:25.870256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:25.871137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:25.871468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:25.871742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] PATH_INFO: `/v3/projects/9623b56d566e4b49ac3def9cdd0d1765` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:25.872940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9623b56d566e4b49ac3def9cdd0d1765)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:25.875568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:25.939582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9623b56d566e4b49ac3def9cdd0d1765', 'request_id': 'req-2a1b71c5-9896-4cda-82bc-9181778614ff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.950176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9623b56d566e4b49ac3def9cdd0d1765', 'request_id': 'req-2a1b71c5-9896-4cda-82bc-9181778614ff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.956904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9623b56d566e4b49ac3def9cdd0d1765', 'request_id': 'req-2a1b71c5-9896-4cda-82bc-9181778614ff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.969705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] The token cache is being invalidate because project 9623b56d566e4b49ac3def9cdd0d1765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:25.969931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.970669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2a1b71c5-9896-4cda-82bc-9181778614ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:25.972705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1009/2052] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:25 2022] DELETE /identity/v3/projects/9623b56d566e4b49ac3def9cdd0d1765 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:25.977137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:25.977488 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:25.977775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:25.998542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:26.000387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:26.066853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1044/2053] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:26.073481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.073855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.074195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.099077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:26.107134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:26.109549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:26.122794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1010/2054] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:26.127017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.138193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176bdd-6308-4035-9990-7546f501b0b3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.143155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.143836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.144117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.144392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.145125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:26.145483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.147096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.160979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1045/2055] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:26.167291 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.177902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108276b-d196-43e3-9751-7fe521bda373 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.184091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.184874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.185160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.185545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.186682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.189826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.223242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1011/2056] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:26.228224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.249601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebf335-dc6f-4b74-9350-78217851ae0d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.309705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.310271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.310555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.310861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.311587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.313757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.338203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1046/2057] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:26.343635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.355786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997a77-47fb-45be-bd2d-7a3325457f1e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.369507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.370328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.370968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.371318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.372123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.373799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.386680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1012/2058] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:26.390791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.414465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2ee86-8c38-479b-b9ea-bdba92d0e9d2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.427877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.429357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.430092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.431191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.438135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.440230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37c23b-af28-46b8-94dd-66d7d6eb7525 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.455010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1047/2059] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:26.458906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.469651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6899ee-1123-469b-acc3-9d73eba402d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.484526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.485570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.485961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.486705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] PATH_INFO: `/v3/projects/5b7f00df57e243688209e63263a1f4db/users/72d1f7985cfa4d75aa167ea06b148004/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.509093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7f00df57e243688209e63263a1f4db, user_id=72d1f7985cfa4d75aa167ea06b148004, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.512297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.528278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1013/2060] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:26 2022] PUT /identity/v3/projects/5b7f00df57e243688209e63263a1f4db/users/72d1f7985cfa4d75aa167ea06b148004/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:26.539214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.539527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.539808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.574698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] MFA Rules not processed for user `72d1f7985cfa4d75aa167ea06b148004`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:26.575969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:26.634658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:26.689065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1048/2061] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:26.693740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.770501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacd5e0-6fa6-4b06-a417-91284cb367f1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.775581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.776262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.776533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.776811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.777584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.779523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.812287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1014/2062] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:26.814603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.831167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5c4f-c8a3-4064-b66b-bed1c15bfca2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:26.928653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:26.929353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:26.929672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:26.930087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:26.930789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:26.932716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:26.969203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1049/2063] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:26 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:26.969311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:26.996019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27524-2a6d-4a32-9389-6cb8e8db6a16 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.007733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.008378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.008813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.013247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.014153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.016039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.038911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1015/2064] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:27.042987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.064705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333d6a2-434a-4149-b3f8-ff53ced59baf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.069946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.070883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.071310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.071735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.072678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.074324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af0a6a-d550-446c-abd3-8c9bd72dc864 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.085232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1050/2065] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:27.089388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.100985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f8e-ed65-484a-8388-25950dcfeb16 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.107407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.109003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.109482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.109903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] PATH_INFO: `/v3/projects/baca2089ceee435f956c4609dff2b75f/users/c01ac98ae48942458a9c3476684d3575/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.129248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=baca2089ceee435f956c4609dff2b75f, user_id=c01ac98ae48942458a9c3476684d3575, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.132843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.149132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1016/2066] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:27 2022] PUT /identity/v3/projects/baca2089ceee435f956c4609dff2b75f/users/c01ac98ae48942458a9c3476684d3575/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:27.159632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.160082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.160494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.191058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] MFA Rules not processed for user `c01ac98ae48942458a9c3476684d3575`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:27.192374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:27.257593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1051/2067] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:27.261938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.314133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:27.325551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555da93-ab5a-4c36-bf9d-f90939242a7d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.330766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.331420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.331708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.331991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.332672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.334600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.360399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1017/2068] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:27 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:27.364577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.375179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f908e47-6ab4-4078-8b8f-e83832cf1140 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.418753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:27.440851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.441571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.441860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.442143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.442859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.444847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.465816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1052/2069] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:27 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:27.470261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.482215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4b63-0a43-4bd7-b513-5187538b3b2a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.486919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.487591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.487883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.488163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.488889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.490522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.502275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1018/2070] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:27.506586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.517157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2114-7875-42c0-a99f-e758c8d6c0a0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.521764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.523396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.523683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.523964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] PATH_INFO: `/v3/projects/bd18ab8176a248269b8380f0a6224df1/users/19b445ee924b4dfbb8f4594e5e1092c8/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.540138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd18ab8176a248269b8380f0a6224df1, user_id=19b445ee924b4dfbb8f4594e5e1092c8, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.544054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.559105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1053/2071] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:27 2022] PUT /identity/v3/projects/bd18ab8176a248269b8380f0a6224df1/users/19b445ee924b4dfbb8f4594e5e1092c8/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:27.563669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.574237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c91a06-544b-4184-83d1-239e2ee3411e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.631748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.632427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.632712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.632987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.633936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.638214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fc321-001a-46ce-aea9-0c8592bb14ef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.650435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1019/2072] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:27.654884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.669759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a373-c047-4447-bc04-1ab47b9b4b07 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.674652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.675548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.675825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.676099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] PATH_INFO: `/v3/projects/bd18ab8176a248269b8380f0a6224df1/users/19b445ee924b4dfbb8f4594e5e1092c8/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.678091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd18ab8176a248269b8380f0a6224df1, user_id=19b445ee924b4dfbb8f4594e5e1092c8, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.680707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.692545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1054/2073] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:27 2022] PUT /identity/v3/projects/bd18ab8176a248269b8380f0a6224df1/users/19b445ee924b4dfbb8f4594e5e1092c8/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:27.715088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.716500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.716996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.755599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] MFA Rules not processed for user `19b445ee924b4dfbb8f4594e5e1092c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:27.757549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:27.868820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1020/2074] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:27.880399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:27.882409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:27.950243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcb896-4598-4cb6-98f8-02f95089c656 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:27.954825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:27.955604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:27.955866 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:27.956187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:27.966616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:27.968171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:27.980926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1055/2075] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:28.908797 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:28.910253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:28.922695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0864f8a-4296-4089-bafc-45caf2f2a523 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:28.926369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:28.927032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:28.927325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:28.927606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:29.004541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:29.021172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:29.022916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e76a1a-cabd-4579-9ec7-dd6386bd2ebb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:29.041961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1021/2076] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:31.816296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:31.817901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:31.900462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ba145-82ef-4ef9-9ad1-fa7ec5161fa4 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:31.905875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:31.906762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:31.907049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:31.907433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] PATH_INFO: `/v3/users/247eb142205647a89d89d4fbe68aca1d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:31.908780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=247eb142205647a89d89d4fbe68aca1d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:31.910834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:31.975351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:31.985530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247eb142205647a89d89d4fbe68aca1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:31.995875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247eb142205647a89d89d4fbe68aca1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.011251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247eb142205647a89d89d4fbe68aca1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.012174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247eb142205647a89d89d4fbe68aca1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.020824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247eb142205647a89d89d4fbe68aca1d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.031610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1056/2077] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:31 2022] DELETE /identity/v3/users/247eb142205647a89d89d4fbe68aca1d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:32.033209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff3884-c3a0-4b5f-8dff-4698f01eb77e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.033811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff3884-c3a0-4b5f-8dff-4698f01eb77e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.033811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff3884-c3a0-4b5f-8dff-4698f01eb77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.055204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eaff3884-c3a0-4b5f-8dff-4698f01eb77e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:32.057180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eaff3884-c3a0-4b5f-8dff-4698f01eb77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.153975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1022/2078] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:32.180548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:32.182097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.266812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a07b0-2b8c-47bb-8f93-e4636bdc4092 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:32.270702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:32.271380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.271772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.272102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.283843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:32.285693 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:32.293019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43f09d-5945-41c8-87d8-19ec5c7a2ccf None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.293427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43f09d-5945-41c8-87d8-19ec5c7a2ccf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.293777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43f09d-5945-41c8-87d8-19ec5c7a2ccf None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.295497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1023/2079] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 14:33:32.297483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1057/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:32.310758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.311358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.311630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.332006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] MFA Rules not processed for user `4a342fa16a564ef38675936d7f51578b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:32.333451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.418546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1024/2081] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:33:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:32.455287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:32.458944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.473160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:32.475985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.480303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f2a86-1d59-46a8-a132-606ca00c58d2 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:32.487372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:32.488558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.489146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.489647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] PATH_INFO: `/v3/users/db6294c41e6844faa4279f5fc5c5360d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.491573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6294c41e6844faa4279f5fc5c5360d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:32.494303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:32.495485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40323e51-8ce7-4441-987c-797867ae5964 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:32.510139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:32.511311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.511768 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.512201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.535070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:32.538021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:32.561516 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1025/2082] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:32.592716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6294c41e6844faa4279f5fc5c5360d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.609281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6294c41e6844faa4279f5fc5c5360d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.629253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6294c41e6844faa4279f5fc5c5360d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.630118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6294c41e6844faa4279f5fc5c5360d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.641967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6294c41e6844faa4279f5fc5c5360d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.650035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1058/2083] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:32 2022] DELETE /identity/v3/users/db6294c41e6844faa4279f5fc5c5360d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:32.669290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:32.671269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.688858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2714f-6194-4434-890c-ccca2e6b1015 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:32.813536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:32.815639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:32.816457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:32.816821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] PATH_INFO: `/v3/projects/ee85aa5a77d8468ab6223adea9777afe` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:32.819204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee85aa5a77d8468ab6223adea9777afe)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:32.821574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:32.838796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:32.899812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee85aa5a77d8468ab6223adea9777afe', 'request_id': 'req-e2393424-7ee2-40d4-9b6a-0de82225fe8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.909102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee85aa5a77d8468ab6223adea9777afe', 'request_id': 'req-e2393424-7ee2-40d4-9b6a-0de82225fe8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.917402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee85aa5a77d8468ab6223adea9777afe', 'request_id': 'req-e2393424-7ee2-40d4-9b6a-0de82225fe8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.931698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] The token cache is being invalidate because project ee85aa5a77d8468ab6223adea9777afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:32.932252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.934237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:32.936012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1026/2084] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:32 2022] DELETE /identity/v3/projects/ee85aa5a77d8468ab6223adea9777afe => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:32.941384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:32.942730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:32.957856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e117eea-c2ed-477c-bd0d-4c7af44c0b70 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:32.990710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:33.034556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:33.035537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:33.035822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:33.036128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] PATH_INFO: `/v3/users/0355504376b94d3db5a1affa8aa67536` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:33.037484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0355504376b94d3db5a1affa8aa67536)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:33.039652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:33.109649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0355504376b94d3db5a1affa8aa67536'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.124716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0355504376b94d3db5a1affa8aa67536'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.145149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0355504376b94d3db5a1affa8aa67536'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.145986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0355504376b94d3db5a1affa8aa67536'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.154361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0355504376b94d3db5a1affa8aa67536'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.162603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1059/2085] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:32 2022] DELETE /identity/v3/users/0355504376b94d3db5a1affa8aa67536 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:33.176161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:33.177783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:33.192885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2393424-7ee2-40d4-9b6a-0de82225fe8d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:33.299460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:33.302339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:33.302799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:33.303099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] PATH_INFO: `/v3/projects/cefc90d2f25d42ec9f9d5567dda60b58` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:33.304418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cefc90d2f25d42ec9f9d5567dda60b58)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:33.306944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:33.379613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cefc90d2f25d42ec9f9d5567dda60b58', 'request_id': 'req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.393075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cefc90d2f25d42ec9f9d5567dda60b58', 'request_id': 'req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.400123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cefc90d2f25d42ec9f9d5567dda60b58', 'request_id': 'req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.432292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] The token cache is being invalidate because project cefc90d2f25d42ec9f9d5567dda60b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:33.432664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.435243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.440119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1027/2086] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:33 2022] DELETE /identity/v3/projects/cefc90d2f25d42ec9f9d5567dda60b58 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:33.444637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:33.460334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:33.483642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8838-7b91-483f-beee-30d599db76c6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:33.567025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:33.568007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:33.568371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:33.568706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] PATH_INFO: `/v3/users/da80b35d27ed4db69b2911af39c29d5e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:33.570494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da80b35d27ed4db69b2911af39c29d5e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:33.572654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:33.595466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:33.597414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:33.645357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96659d-e0f3-4a6d-9d44-a7c3ba97ebba admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:33.658593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:33.659686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:33.660002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:33.660179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] PATH_INFO: `/v3/projects/31710c3ca48746c1b73142795b5f15d0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:33.667550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=31710c3ca48746c1b73142795b5f15d0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:33.670551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:33.674442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da80b35d27ed4db69b2911af39c29d5e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.679456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:33.688920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da80b35d27ed4db69b2911af39c29d5e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.713465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da80b35d27ed4db69b2911af39c29d5e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.714439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da80b35d27ed4db69b2911af39c29d5e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.727149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6f6696ae-616f-4172-ba69-36c245944139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da80b35d27ed4db69b2911af39c29d5e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.737299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1060/2087] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:33 2022] DELETE /identity/v3/users/da80b35d27ed4db69b2911af39c29d5e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:33.743960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:33.770286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31710c3ca48746c1b73142795b5f15d0', 'request_id': 'req-c03b416b-ef6b-4d33-94e2-d9281099c63e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.790527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31710c3ca48746c1b73142795b5f15d0', 'request_id': 'req-c03b416b-ef6b-4d33-94e2-d9281099c63e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.799051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31710c3ca48746c1b73142795b5f15d0', 'request_id': 'req-c03b416b-ef6b-4d33-94e2-d9281099c63e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.814142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] The token cache is being invalidate because project 31710c3ca48746c1b73142795b5f15d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:33.814724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.817098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c03b416b-ef6b-4d33-94e2-d9281099c63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:33.819424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1028/2088] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:33 2022] DELETE /identity/v3/projects/31710c3ca48746c1b73142795b5f15d0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:33.824289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:33.824819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:33.825102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:33.868189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:33.869484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.016945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1061/2089] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:34.018264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.018561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.018848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.052594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:34.054837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.069113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1029/2090] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:34.073658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.085474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343fac-331c-47a4-86e5-e2337fc56e4c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.091163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.092424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.092785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.093136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.093983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:34.094368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.096312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.113667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1062/2091] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:34.118175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.128805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6b4c8-f83b-4574-a84d-800455f09fce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.135427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.136109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.136396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.136677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.137424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.139584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.201474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1030/2092] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:34.202518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.220048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.221984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.226365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce0854-dac4-4feb-97d9-c037f80aa35f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.333735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568cd83-9844-4583-997f-c303498fe695 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.340543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.342965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.345919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.347426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] PATH_INFO: `/v3/projects/ace343fcc26740f7a080dec03d9bab44` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.349673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace343fcc26740f7a080dec03d9bab44)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.354975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.365534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.366275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.366564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.366887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.367582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.372145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.409186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1063/2093] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:34.413656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.428138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2d486-d3dd-4d9a-9237-6df1dcb55bc8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.433697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.434323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.434901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.435805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.436592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.438394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.454906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1064/2094] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:34.462273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.474908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837bd18-1b23-474d-a3d7-b783efbb042a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.482839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace343fcc26740f7a080dec03d9bab44', 'request_id': 'req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:34.494446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace343fcc26740f7a080dec03d9bab44', 'request_id': 'req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:34.502510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace343fcc26740f7a080dec03d9bab44', 'request_id': 'req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:34.516888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] The token cache is being invalidate because project ace343fcc26740f7a080dec03d9bab44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:34.517539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:34.518366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:34.519396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:34.520300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1031/2095] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:34 2022] DELETE /identity/v3/projects/ace343fcc26740f7a080dec03d9bab44 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:34.521204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.522833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.549130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:34.557836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.559098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.559519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.559840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.560837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.563229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa3be1-4e80-4b85-abaa-20ae21f29f08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.574667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1065/2096] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:34.575984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.576335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.576683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.595547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:34.598601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.609147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab43b7-43c0-4ed1-af07-44fa4995d42e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.614114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.615156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.615512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.615891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.617893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.624263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1066/2097] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:34.624263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.625881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.638251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c110824-ce6a-43f5-a285-9661f22deaa1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.645104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.646274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.646558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.646877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] PATH_INFO: `/v3/projects/96c7936d5e6b42399dea99c86af8ac52/users/a2f7e43ef8a24184916ab322479dbf71/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.666355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c7936d5e6b42399dea99c86af8ac52, user_id=a2f7e43ef8a24184916ab322479dbf71, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.669066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c86e5-3dce-4246-8ad7-bc63d42071fb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.690727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1067/2098] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:34 2022] PUT /identity/v3/projects/96c7936d5e6b42399dea99c86af8ac52/users/a2f7e43ef8a24184916ab322479dbf71/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:34.692490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.692783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.693949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.717439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.720364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e050b3b-03ae-4a15-941b-38954cc957fb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.725259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:34.726729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.735081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1032/2099] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:34.736454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.736736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.737180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.775192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] MFA Rules not processed for user `a2f7e43ef8a24184916ab322479dbf71`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:34.776892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:34.817507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1068/2100] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:34.822815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.843427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3939d-635c-4e94-b847-07d7e0d41ab5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.850157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.851107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.851486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.852014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.853458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:34.853868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.856178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.878747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1069/2101] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:34.882042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.891998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1033/2102] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:34.895884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.900857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccd5eb-0172-4529-b139-be46f9fd1e2d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.911770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b080626-9f31-49ce-8ff0-19b1279667aa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:34.918265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.919006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.919296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.919576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.920323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.923557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.926324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:34.927175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:34.928082 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:34.928512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:34.929402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:34.931631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:34.954497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1070/2103] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:34.958543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:34.962577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1034/2104] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:34.966855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.018281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.044849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ceb4c-5710-4dfd-bea8-1cec8b0acaa9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.050628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.051374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.051656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.051935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.052731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.054852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.067634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3017f-3d2b-45fc-b80b-abd4a2b3a1ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.074996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.075803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.076046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.076321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.076760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1035/2105] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:35.077157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.079425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.081538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.089934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.091459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344228-7713-4bf9-a82e-35b3466b5fef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.098088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.099191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.099541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.099812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.100695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.103120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.106298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1071/2106] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:34 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:35.108719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.118143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1036/2107] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:35.119414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.123000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc915dc8-98a8-47f6-88a8-e37443c5182b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.129377 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.130403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.130561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184b3e0-ecca-4ec5-9a44-3cdf6d7f7d3a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.130814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.131163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.131940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.133561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.135286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.136120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.136411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.136832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.137590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.139164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.149389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1072/2108] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:35.150442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.153363 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1037/2109] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:35.154556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.163267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29e591-b5c2-4554-9381-ae276d7ef619 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.164194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36b070-ebb4-46bb-bd05-bfb5be63dc03 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.167907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.168560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.168771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.168879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.169109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.169691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.169868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.169973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.170236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] PATH_INFO: `/v3/projects/10e9a8734fe041eb94e9f544878f76e5/users/823497e2a9744fef8d658eafbe59094e/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.171490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.182448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1073/2110] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:35.185734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e9a8734fe041eb94e9f544878f76e5, user_id=823497e2a9744fef8d658eafbe59094e, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.187430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.188919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0d6a-222c-4ef0-9a78-47f23dfe21dd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.197629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d64cc-fb89-4a67-b6a2-3b582e6a7184 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.203048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1038/2111] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:35 2022] PUT /identity/v3/projects/10e9a8734fe041eb94e9f544878f76e5/users/823497e2a9744fef8d658eafbe59094e/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:35.213794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.214088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.214374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.242030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] MFA Rules not processed for user `823497e2a9744fef8d658eafbe59094e`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:35.243253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.266012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.266904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.267690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.267963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] PATH_INFO: `/v3/projects/3fbf264975604213bdf87a92d53b158c/users/9777ffe5b098462892ed411694784d4c/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.281871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbf264975604213bdf87a92d53b158c, user_id=9777ffe5b098462892ed411694784d4c, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.284791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c42eb6-58ee-4a4f-b7fe-29b87f6f0e79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.298003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1074/2112] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:35 2022] PUT /identity/v3/projects/3fbf264975604213bdf87a92d53b158c/users/9777ffe5b098462892ed411694784d4c/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:35.308708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.309149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.309465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.313732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1039/2113] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:35.318074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.329072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11c2ef-69e2-4ea1-856e-2d1eadda0ab6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.338221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] MFA Rules not processed for user `9777ffe5b098462892ed411694784d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:35.339631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.387785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.388444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.388733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.389023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.389824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.391805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.405720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.412393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1075/2114] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:35.423278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1040/2115] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:35.423601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.424535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.424997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.488275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.492079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532753be-826e-4868-aa4f-a6edc8c85395 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.494407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6c39a-174c-4f97-b5ee-35b075fec84a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.497495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.498164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.498446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.499324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.500060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.502047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.525531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1041/2116] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:35.529326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.539727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ae3-6fdf-42db-9fff-2e298a7e19b9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.544899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.546522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.546809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.547076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.547798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.549425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.550297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.551015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.551406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.551701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.560286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1042/2117] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:35.565148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.575608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c51182-fbf7-49aa-96a4-3f6e1be18d05 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.582862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.583792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.584076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.584354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] PATH_INFO: `/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff/users/6bc9c14ed3be40d3aae0bbbb4250442f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.602915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f26644d63fa4eee8edfe5f6916ec7ff, user_id=6bc9c14ed3be40d3aae0bbbb4250442f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.606211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.612567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.614376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.624859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1076/2118] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:35.625549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1043/2119] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:35 2022] PUT /identity/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff/users/6bc9c14ed3be40d3aae0bbbb4250442f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:35.630388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.642762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0140c-d921-421e-a1c7-0e6521b0465f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.699458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.700154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.700423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.700697 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.701493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.703673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558f4ca-c9a2-4684-b809-c8e74df2e6ed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.713932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1077/2120] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:35.718732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:35.729150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78f2a-c92f-4791-99ac-3db70ff4fbda admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:35.735985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:35.737006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.737389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.737767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] PATH_INFO: `/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff/users/6bc9c14ed3be40d3aae0bbbb4250442f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.740013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f26644d63fa4eee8edfe5f6916ec7ff, user_id=6bc9c14ed3be40d3aae0bbbb4250442f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:35.742695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:35.757416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1044/2121] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:35 2022] PUT /identity/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff/users/6bc9c14ed3be40d3aae0bbbb4250442f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:35.765677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:35.765989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:35.766318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:35.796954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] MFA Rules not processed for user `6bc9c14ed3be40d3aae0bbbb4250442f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:35.798890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:35.849037 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:35.891089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1078/2122] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:35.958547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:36.019738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f150d-1a3b-46c5-8a56-0d2607cb51e6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:36.023367 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:36.024007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:36.024294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:36.024569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:36.034295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:36.035771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:36.046580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1045/2123] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:36.152497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:36.162413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4bbe0-0a38-4108-8ec5-be69d6da361a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:36.165887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:36.166524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:36.166808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:36.167086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:36.223451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:36.224880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a18c8-5da9-4aa6-9558-dc73a01279a7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:36.234206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1079/2124] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:42.144384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:42.146018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:42.195416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:42.217638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206130f7-9ec1-453e-b3f0-d0dcccb41f7d service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:42.223685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:42.224554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:42.224837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:42.225119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] PATH_INFO: `/v3/users/72d1f7985cfa4d75aa167ea06b148004` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:42.226339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d1f7985cfa4d75aa167ea06b148004)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:42.228253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:42.274798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d1f7985cfa4d75aa167ea06b148004'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.282545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d1f7985cfa4d75aa167ea06b148004'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.283173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d1f7985cfa4d75aa167ea06b148004'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.295180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d1f7985cfa4d75aa167ea06b148004'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.301945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d1f7985cfa4d75aa167ea06b148004'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.307834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1046/2125] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:42 2022] DELETE /identity/v3/users/72d1f7985cfa4d75aa167ea06b148004 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:42.312715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:42.313185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:42.313301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:42.330383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:42.331568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:42.391797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1080/2126] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:42.401295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:42.402814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:42.460580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8b4d-dcd5-4b19-b42d-2e61a28c9027 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:42.464416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:42.465246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:42.465372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:42.465640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:42.475631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:42.477211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:42.487476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1047/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:42.754040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:42.755466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:42.771858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90ff32-2431-4d9d-a163-5be51ef2a936 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:42.776687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:42.777717 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:42.777996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:42.778254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] PATH_INFO: `/v3/projects/5b7f00df57e243688209e63263a1f4db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:42.779461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7f00df57e243688209e63263a1f4db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:42.781955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:42.828882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7f00df57e243688209e63263a1f4db', 'request_id': 'req-9c3146ad-2999-44f5-93f2-ea7b473cbb40'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.836242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7f00df57e243688209e63263a1f4db', 'request_id': 'req-9c3146ad-2999-44f5-93f2-ea7b473cbb40'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.841899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7f00df57e243688209e63263a1f4db', 'request_id': 'req-9c3146ad-2999-44f5-93f2-ea7b473cbb40'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.850064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:42.854133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] The token cache is being invalidate because project 5b7f00df57e243688209e63263a1f4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:42.854606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.855657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.857326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1081/2128] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:42 2022] DELETE /identity/v3/projects/5b7f00df57e243688209e63263a1f4db => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:42.861230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:42.871482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577664c-a468-43ae-89f8-6d86c613dc0f service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:42.933597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:42.934462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:42.934741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:42.935023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] PATH_INFO: `/v3/users/c01ac98ae48942458a9c3476684d3575` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:42.936272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01ac98ae48942458a9c3476684d3575)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:42.938234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:42.940611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:42.988373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01ac98ae48942458a9c3476684d3575'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.996109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01ac98ae48942458a9c3476684d3575'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:42.996716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01ac98ae48942458a9c3476684d3575'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.008691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01ac98ae48942458a9c3476684d3575'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.016607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01ac98ae48942458a9c3476684d3575'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.022844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1048/2129] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:42 2022] DELETE /identity/v3/users/c01ac98ae48942458a9c3476684d3575 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:43.291452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:43.292983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.310592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3146ad-2999-44f5-93f2-ea7b473cbb40 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:43.367508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:43.368383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.368670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.368945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] PATH_INFO: `/v3/projects/baca2089ceee435f956c4609dff2b75f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.370212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=baca2089ceee435f956c4609dff2b75f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:43.372128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:43.420052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baca2089ceee435f956c4609dff2b75f', 'request_id': 'req-bb982c46-4a39-4d05-8cb6-62e8f11727ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.428046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baca2089ceee435f956c4609dff2b75f', 'request_id': 'req-bb982c46-4a39-4d05-8cb6-62e8f11727ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.434095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baca2089ceee435f956c4609dff2b75f', 'request_id': 'req-bb982c46-4a39-4d05-8cb6-62e8f11727ba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.445432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] The token cache is being invalidate because project baca2089ceee435f956c4609dff2b75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:43.445719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.446341 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.447860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1082/2130] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:43 2022] DELETE /identity/v3/projects/baca2089ceee435f956c4609dff2b75f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:43.451906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:43.453420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.466085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834166d-9d78-4994-9cc9-7654757c1db6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:43.526112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:43.526977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.527261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.527540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] PATH_INFO: `/v3/users/19b445ee924b4dfbb8f4594e5e1092c8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.528727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b445ee924b4dfbb8f4594e5e1092c8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:43.530829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:43.584687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b445ee924b4dfbb8f4594e5e1092c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.591936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b445ee924b4dfbb8f4594e5e1092c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.592555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b445ee924b4dfbb8f4594e5e1092c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.604410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b445ee924b4dfbb8f4594e5e1092c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.611933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-89bbf3aa-1e13-4d2e-be65-972ed8ac0660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b445ee924b4dfbb8f4594e5e1092c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:43.614060 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:43.618355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1049/2131] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:43 2022] DELETE /identity/v3/users/19b445ee924b4dfbb8f4594e5e1092c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:43.883347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:43.884803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:43.902359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982c46-4a39-4d05-8cb6-62e8f11727ba admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:43.962622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:43.963495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:43.963775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:43.964047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] PATH_INFO: `/v3/projects/bd18ab8176a248269b8380f0a6224df1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:43.965240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd18ab8176a248269b8380f0a6224df1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:43.968806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.005071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:44.018419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd18ab8176a248269b8380f0a6224df1', 'request_id': 'req-b0be9a24-7ee6-48d6-a814-1e19290a82e8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.029532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd18ab8176a248269b8380f0a6224df1', 'request_id': 'req-b0be9a24-7ee6-48d6-a814-1e19290a82e8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.035590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd18ab8176a248269b8380f0a6224df1', 'request_id': 'req-b0be9a24-7ee6-48d6-a814-1e19290a82e8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.046876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] The token cache is being invalidate because project bd18ab8176a248269b8380f0a6224df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:44.047159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.047786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0be9a24-7ee6-48d6-a814-1e19290a82e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.049314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1083/2132] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:43 2022] DELETE /identity/v3/projects/bd18ab8176a248269b8380f0a6224df1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:44.055346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.055633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.055911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.077052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:44.078485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:44.140535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1050/2133] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:44.149047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.149381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.149670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.171213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:44.172427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:44.180353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1084/2134] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:44.184997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.196633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c1b13-5b1a-43c0-99f4-a2f1b89acbfe None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.201471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.202934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.203209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.203483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.204220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:44.204527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.206246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.218606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1051/2135] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:44.229502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.239277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd368cb-a06a-46f2-a233-9835e9eadc58 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.244138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.244830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.245112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.245428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.246126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.248026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.275189 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1085/2136] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:44.282844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.292964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886726d6-9add-46f0-b65a-246e0b4df9f3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.351568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.352229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.352510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.352783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.353501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.355415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.379938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1052/2137] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:44.381053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.391203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127383c-77fb-44c4-8b7a-86da5016e696 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.396054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.396710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.396994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.397269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.398028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.399604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.410361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1086/2138] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:44.414141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.425152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa1392-21f4-461c-8655-61bd0296072c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.429915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.430574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.430862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.431143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.431866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.433729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1b061-df6c-4a90-8738-57e343e9d94b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.444237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1053/2139] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:44.448484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.458537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95e10-c55c-4fdd-ba3d-16cc9689a79a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.463077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.463958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.464287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.464616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] PATH_INFO: `/v3/projects/523352456b5f4223b61fe445c322fe54/users/daaeca2d7a854731b6558cc37fdbe0a7/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.479015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523352456b5f4223b61fe445c322fe54, user_id=daaeca2d7a854731b6558cc37fdbe0a7, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.481664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.495767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1087/2140] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:44 2022] PUT /identity/v3/projects/523352456b5f4223b61fe445c322fe54/users/daaeca2d7a854731b6558cc37fdbe0a7/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:44.512606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.513134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.513493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.547729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] MFA Rules not processed for user `daaeca2d7a854731b6558cc37fdbe0a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:44.549004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:44.615785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:44.630937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1054/2141] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:44.661762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.663489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:44.719873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d952b2-c1df-4485-b7d0-53149cf2e726 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.723580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.724223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.724532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.724815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.733575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.734995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf6cee-3cd0-401e-bb86-d034055cdff1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.743698 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1088/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:44.758909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:44.820573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d93aa7-2b2a-4465-9eac-bedf64fee5b8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:44.825003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:44.825885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.826169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.826445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] PATH_INFO: `/v3/users/daaeca2d7a854731b6558cc37fdbe0a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.827651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daaeca2d7a854731b6558cc37fdbe0a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:44.829888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:44.876352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daaeca2d7a854731b6558cc37fdbe0a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.883092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daaeca2d7a854731b6558cc37fdbe0a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.883745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daaeca2d7a854731b6558cc37fdbe0a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.896742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daaeca2d7a854731b6558cc37fdbe0a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.904585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daaeca2d7a854731b6558cc37fdbe0a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:44.911681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1055/2143] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:44 2022] DELETE /identity/v3/users/daaeca2d7a854731b6558cc37fdbe0a7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:44.916569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:44.916926 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:44.917234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:44.936523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:44.937871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:44.989416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:45.001039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1089/2144] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:45.010471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:45.011889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:45.068390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1aef1d-b9a9-4c97-b125-c39671a94603 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:45.071988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:45.072623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.072903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.073179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.083719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:45.085193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:45.095484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1056/2145] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:45.369000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:45.370489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:45.393372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1ce5-1e83-4e1b-ad14-61cf4787b800 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:45.395141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:45.396531 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:45.415532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:45.417283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.417786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.418275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] PATH_INFO: `/v3/projects/523352456b5f4223b61fe445c322fe54` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.421200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523352456b5f4223b61fe445c322fe54)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:45.423897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:45.428614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779abaf4-2d7a-4452-870b-57b45cfcc969 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:45.448675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:45.449936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.450293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.450657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] PATH_INFO: `/v3/users/a2f7e43ef8a24184916ab322479dbf71` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.452070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f7e43ef8a24184916ab322479dbf71)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:45.454286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:45.471802 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:45.501850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523352456b5f4223b61fe445c322fe54', 'request_id': 'req-5412874f-a16c-419a-a5da-e329986814e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.511061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523352456b5f4223b61fe445c322fe54', 'request_id': 'req-5412874f-a16c-419a-a5da-e329986814e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.519617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f7e43ef8a24184916ab322479dbf71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.520080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523352456b5f4223b61fe445c322fe54', 'request_id': 'req-5412874f-a16c-419a-a5da-e329986814e3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.531388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f7e43ef8a24184916ab322479dbf71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.532270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f7e43ef8a24184916ab322479dbf71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.533782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] The token cache is being invalidate because project 523352456b5f4223b61fe445c322fe54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:45.534064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.534727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5412874f-a16c-419a-a5da-e329986814e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.536263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1090/2146] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:45 2022] DELETE /identity/v3/projects/523352456b5f4223b61fe445c322fe54 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:45.545718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f7e43ef8a24184916ab322479dbf71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.554911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f7e43ef8a24184916ab322479dbf71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:45.565389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1057/2147] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:45 2022] DELETE /identity/v3/users/a2f7e43ef8a24184916ab322479dbf71 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:45.569792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.570072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.570349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.592908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:45.605662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:45.744696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1091/2148] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:45.757201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:45.759627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:45.824572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d7dee-82bc-4277-8b5c-a4760fe9e04b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:45.828752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:45.829574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:45.830471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:45.830471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:45.840787 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:45.842406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:45.863934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1058/2149] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:46.314067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:46.315479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:46.327400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d29d03-07d8-4803-9f30-08746a5aba37 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:46.333658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:46.334566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:46.334850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:46.335133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] PATH_INFO: `/v3/projects/96c7936d5e6b42399dea99c86af8ac52` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:46.337096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c7936d5e6b42399dea99c86af8ac52)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:46.339775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:46.396045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c7936d5e6b42399dea99c86af8ac52', 'request_id': 'req-92f97320-c63b-4453-9864-55301601af7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.405846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c7936d5e6b42399dea99c86af8ac52', 'request_id': 'req-92f97320-c63b-4453-9864-55301601af7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.415882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c7936d5e6b42399dea99c86af8ac52', 'request_id': 'req-92f97320-c63b-4453-9864-55301601af7b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.437868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] The token cache is being invalidate because project 96c7936d5e6b42399dea99c86af8ac52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:46.438472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.439165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.441248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1092/2150] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:46 2022] DELETE /identity/v3/projects/96c7936d5e6b42399dea99c86af8ac52 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:46.445466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:46.455757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8742c-5146-47b8-9b13-55d0cdb63b78 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:46.536400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:46.537500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:46.537889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:46.538294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] PATH_INFO: `/v3/users/823497e2a9744fef8d658eafbe59094e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:46.539666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=823497e2a9744fef8d658eafbe59094e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:46.541769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:46.587688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:46.602810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '823497e2a9744fef8d658eafbe59094e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.613731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '823497e2a9744fef8d658eafbe59094e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.614450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '823497e2a9744fef8d658eafbe59094e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.627612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '823497e2a9744fef8d658eafbe59094e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.634879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '823497e2a9744fef8d658eafbe59094e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:46.641374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1059/2151] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:46 2022] DELETE /identity/v3/users/823497e2a9744fef8d658eafbe59094e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:46.962257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:46.963787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:46.984667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f97320-c63b-4453-9864-55301601af7b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:46.989759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:47.078476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:47.079347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.079628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.079900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] PATH_INFO: `/v3/projects/10e9a8734fe041eb94e9f544878f76e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.081072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e9a8734fe041eb94e9f544878f76e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:47.082998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:47.130749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e9a8734fe041eb94e9f544878f76e5', 'request_id': 'req-b35ddff5-bdd8-4f87-a053-7b6661fcff36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.139794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e9a8734fe041eb94e9f544878f76e5', 'request_id': 'req-b35ddff5-bdd8-4f87-a053-7b6661fcff36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.145987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e9a8734fe041eb94e9f544878f76e5', 'request_id': 'req-b35ddff5-bdd8-4f87-a053-7b6661fcff36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.157782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] The token cache is being invalidate because project 10e9a8734fe041eb94e9f544878f76e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:47.158074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.158882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.160604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1093/2152] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:46 2022] DELETE /identity/v3/projects/10e9a8734fe041eb94e9f544878f76e5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:47.164756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:47.166284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:47.178954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eaae0-c8a2-4aa1-8894-6366a2521cb2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:47.237085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:47.237996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.238285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.238569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] PATH_INFO: `/v3/users/6bc9c14ed3be40d3aae0bbbb4250442f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.239840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc9c14ed3be40d3aae0bbbb4250442f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:47.241774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:47.306950 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc9c14ed3be40d3aae0bbbb4250442f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.316997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc9c14ed3be40d3aae0bbbb4250442f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.317846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc9c14ed3be40d3aae0bbbb4250442f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.333891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc9c14ed3be40d3aae0bbbb4250442f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.344010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a6d060a-f075-4027-9881-ddb21d3d004d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc9c14ed3be40d3aae0bbbb4250442f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.351522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1060/2153] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:47 2022] DELETE /identity/v3/users/6bc9c14ed3be40d3aae0bbbb4250442f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:47.661879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:47.664206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:47.692770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddff5-bdd8-4f87-a053-7b6661fcff36 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:47.752482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:47.753361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.753735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.754013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] PATH_INFO: `/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.755220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f26644d63fa4eee8edfe5f6916ec7ff)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:47.757109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:47.796684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:47.804838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f26644d63fa4eee8edfe5f6916ec7ff', 'request_id': 'req-c09e558d-38e7-4adc-b27b-baced34fe864'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.812449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f26644d63fa4eee8edfe5f6916ec7ff', 'request_id': 'req-c09e558d-38e7-4adc-b27b-baced34fe864'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.818034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f26644d63fa4eee8edfe5f6916ec7ff', 'request_id': 'req-c09e558d-38e7-4adc-b27b-baced34fe864'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.828898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] The token cache is being invalidate because project 0f26644d63fa4eee8edfe5f6916ec7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:47.829186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.829903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c09e558d-38e7-4adc-b27b-baced34fe864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:47.831537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1094/2154] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:47 2022] DELETE /identity/v3/projects/0f26644d63fa4eee8edfe5f6916ec7ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:47.837451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.837905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.837996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.857482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:47.859305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:47.884696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:47.919013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1061/2155] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:47.927715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.928009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.928297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.947699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:47.948900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:47.960387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1095/2156] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:47.961453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:47.971814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802300c-a8a1-4f44-8678-d5ce7fb367c6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:47.976470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:47.977120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:47.977439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:47.977715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:47.978435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:47.978742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:47.980443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:47.992742 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1062/2157] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:33:47.997392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.007113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372c810-ff1e-495d-a114-5b1a75e7f94f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.011749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.012395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.012679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.012958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.013691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.016263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.043142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1096/2158] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:47 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:48.048697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.059740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6e6a3-f30b-4687-aab6-94067755b886 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.119585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.120257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.120535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.120811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.121538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.124171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.143046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1063/2159] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:48.147624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.157955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c680-a607-40ed-8bcf-76d98f599efb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.162876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.163553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.163825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.164104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.164857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.166513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.177888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1097/2160] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:48.182546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.193079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab9451-a5b0-45b6-927e-ae1a516d9e35 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.197913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.198578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.198869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.199143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.199864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.202027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6268c-6648-425a-9225-15527308e78b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.216202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1064/2161] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:48.218166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.230691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a5cd-b2d5-4694-93fb-02ef33e48939 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.235981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.236865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.237147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.237452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] PATH_INFO: `/v3/projects/0163d466aa594f58a480d7c362f3caa8/users/1b0cfc8c3a8d41bdada3cb9888648e64/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.252233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0163d466aa594f58a480d7c362f3caa8, user_id=1b0cfc8c3a8d41bdada3cb9888648e64, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.254931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.268953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1098/2162] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:48 2022] PUT /identity/v3/projects/0163d466aa594f58a480d7c362f3caa8/users/1b0cfc8c3a8d41bdada3cb9888648e64/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:48.280368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.280754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.281099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.309445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] MFA Rules not processed for user `1b0cfc8c3a8d41bdada3cb9888648e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:48.310731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.376394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1065/2163] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:48.380840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.430059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.431672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.455259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc84b6-a287-430b-9eaa-6644b567b077 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.460099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.460787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.461066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.461371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.462064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.463996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.472505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:48.500857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1099/2164] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:48.502644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.515667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36909a-5c1e-4226-aee1-9ea02eae1438 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.556367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:48.575094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2984-171f-43bb-a22e-262f9b53762d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.579852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.580555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.580821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.581114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.581820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.583198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.583768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.584229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.584519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.584790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] PATH_INFO: `/v3/users/9777ffe5b098462892ed411694784d4c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.586036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9777ffe5b098462892ed411694784d4c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.587925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.606941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1100/2165] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:48.612258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.623885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d18-4421-4553-a6f2-2b377ad6f6cb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.629135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.629876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.630164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.630552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.631286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.633389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.644011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1101/2166] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:48.648155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.653677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9777ffe5b098462892ed411694784d4c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:48.662882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ae9b-b92d-4abc-be31-b024275e5530 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.663385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9777ffe5b098462892ed411694784d4c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:48.664197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9777ffe5b098462892ed411694784d4c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:48.669219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.673122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.673447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.673729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] PATH_INFO: `/v3/projects/8bd538a356c147629432c2e3cf713b10/users/5e3ec13327dc48b2ace67e1819b50891/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.679848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9777ffe5b098462892ed411694784d4c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:48.688278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1ac64ed7-651d-49d9-baac-35df78930029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9777ffe5b098462892ed411694784d4c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:48.692537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd538a356c147629432c2e3cf713b10, user_id=5e3ec13327dc48b2ace67e1819b50891, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.697169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.697946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1066/2167] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:48 2022] DELETE /identity/v3/users/9777ffe5b098462892ed411694784d4c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:48.705261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.705759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.706225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.718524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1102/2168] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:48 2022] PUT /identity/v3/projects/8bd538a356c147629432c2e3cf713b10/users/5e3ec13327dc48b2ace67e1819b50891/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:48.724131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.725579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.728725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:48.730127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.821015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb726-125c-40c6-9c3c-d2cd690095b9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.833409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.835597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.837058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:48.838852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.840303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.841918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.847471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.859903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1067/2169] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:48.863622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1103/2170] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:48.872800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.874138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:48.877911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:48.890476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c246b-c865-4eab-82bb-9c3dd9c8a180 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.890476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:48.890476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.890476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.890859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] PATH_INFO: `/v3/projects/8bd538a356c147629432c2e3cf713b10/users/5e3ec13327dc48b2ace67e1819b50891/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.892897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd538a356c147629432c2e3cf713b10, user_id=5e3ec13327dc48b2ace67e1819b50891, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:48.897195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19064c7-c470-4521-9725-d7fe8a7efe56 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:48.913379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1068/2171] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:48 2022] PUT /identity/v3/projects/8bd538a356c147629432c2e3cf713b10/users/5e3ec13327dc48b2ace67e1819b50891/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:48.922017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:48.922628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:48.922921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:48.956632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfee1a3-8869-494a-967e-d2a61fc2041e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:48.978148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] MFA Rules not processed for user `5e3ec13327dc48b2ace67e1819b50891`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:48.979995 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.019452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.020149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.020438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.020719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.029651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:49.069839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1069/2172] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:49.082936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.084476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.090360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.091815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.102186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1104/2173] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:49.144509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a222867-ffa9-4776-bf64-4c8092bf09e0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.148496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.149293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.149718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.150096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.160150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.162225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.176444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1070/2174] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:49.261470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.271473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0da3c-3ae0-4420-8ad0-a3c47caa8cd5 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.275176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.275824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.276129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.276387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.278244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.341810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.343887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37f281-a0d1-4441-bc45-d3d9162a33d1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.354229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1105/2175] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:49.421701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.423165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.440762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29845b6-7ac8-4bc9-baa6-9538a93ab639 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.446365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.447423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.447860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.448141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] PATH_INFO: `/v3/projects/3fbf264975604213bdf87a92d53b158c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.449721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbf264975604213bdf87a92d53b158c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.452554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.520605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbf264975604213bdf87a92d53b158c', 'request_id': 'req-77b1810a-f276-41cc-89d0-9d2597ad17fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:49.532980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbf264975604213bdf87a92d53b158c', 'request_id': 'req-77b1810a-f276-41cc-89d0-9d2597ad17fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:49.542827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:49.544667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbf264975604213bdf87a92d53b158c', 'request_id': 'req-77b1810a-f276-41cc-89d0-9d2597ad17fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:49.566754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] The token cache is being invalidate because project 3fbf264975604213bdf87a92d53b158c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:49.567216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:49.568293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-77b1810a-f276-41cc-89d0-9d2597ad17fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:49.573973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1071/2176] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:49 2022] DELETE /identity/v3/projects/3fbf264975604213bdf87a92d53b158c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:49.578821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.579116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.579400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.612732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:49.614243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.702486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1106/2177] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:49.710917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.711309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.711744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.737086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:49.738357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:49.746341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1072/2178] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:49.750751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.761106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078c8bd-ca88-4f5d-aec2-0f157b2b14b5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.777241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.777988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.778280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.778567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.780193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:49.780416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.782133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.810559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1107/2179] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:49.811819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.822347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf350-5567-4835-a8ca-892c7741e1d6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.827736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.828393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.828679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.828959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.829713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.832374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.857555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1073/2180] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:49 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:49.861830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.871751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3859429-7348-4c70-99d9-727d28a6ea4e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.900617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:49.931166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.931166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.931354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.931548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.932238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.934445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:49.960890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1108/2181] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:49 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:49.961888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:49.973595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88039-21da-49bb-98fa-ffcb8a5a0367 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:49.993779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:49.994494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:49.994780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:49.995090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:49.995829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:49.997462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.010572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1074/2182] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:50.015235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.026777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3ce18-10e3-4dd3-ae7a-4d3c22aa6c6b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.032077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.032933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.033319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.033741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.034576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.036273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aa903-87b9-4c97-85a4-85c3d333d2f5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.049493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1109/2183] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:50.053881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.065225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb7cf-ce07-4b03-9e24-32980aee6ec5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.070464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.071616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.071902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.072193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] PATH_INFO: `/v3/projects/9237174420004af8add8d53ec8a45aad/users/adddeb29f55e43789fe92ef73800caa2/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.072469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.072760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.073038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.091229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9237174420004af8add8d53ec8a45aad, user_id=adddeb29f55e43789fe92ef73800caa2, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.094265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:50.094460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af92b1-2f3b-4806-98b2-ae355703c23e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.096518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:50.108662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1075/2184] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:50 2022] PUT /identity/v3/projects/9237174420004af8add8d53ec8a45aad/users/adddeb29f55e43789fe92ef73800caa2/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:50.119920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.120277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.120655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.149209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] MFA Rules not processed for user `adddeb29f55e43789fe92ef73800caa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:50.150575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:50.157276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1110/2185] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:33:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:50.199198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.201105 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:50.221035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1076/2186] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:50.226312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.277555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f86dba-d7f2-4ad4-b7e2-59f5f9c32ff4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.288244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15afce4-bcb2-42aa-a3b6-f4f2b6ec0651 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.293092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.293913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.294337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.294675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.295535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.297608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.337086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:50.340076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1077/2187] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:50 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:50.345374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.352781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:50.357042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.358977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.359374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.359780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.431677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca2d39-da88-43a7-863a-d4d1713f791b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.437325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.438046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.438334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.438613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.439314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.439569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.441470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.441611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.454163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1111/2188] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:33:50.469312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.475384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1078/2189] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:50 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:50.483128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771c14d-96d7-4b83-8706-8c0142ecf70d service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.488173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.489092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.489531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.489690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.490482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.492008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.504533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1112/2190] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:50.508557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.519053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bf273-123c-44d5-b02c-751b15111586 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.524004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.525080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.525491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.525861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] PATH_INFO: `/v3/projects/122ce77e87bf437bbab55556fbf3b73e/users/ba2f4fdae3f54f2f841077b1789fa530/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.542302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=122ce77e87bf437bbab55556fbf3b73e, user_id=ba2f4fdae3f54f2f841077b1789fa530, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.545032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.573220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1079/2191] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:50 2022] PUT /identity/v3/projects/122ce77e87bf437bbab55556fbf3b73e/users/ba2f4fdae3f54f2f841077b1789fa530/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:50.574767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.587905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd3ede-ec33-4e08-9533-f914ae101358 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.676734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.677491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.677777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.678061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.678795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.680438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059b000-d5d1-4dab-969d-93c30bcbff92 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.693513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1113/2192] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:50.696578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.707811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c480c-cf2f-41e9-8eaf-7613704d6675 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:50.716201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:50.717971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.718407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.718844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] PATH_INFO: `/v3/projects/122ce77e87bf437bbab55556fbf3b73e/users/ba2f4fdae3f54f2f841077b1789fa530/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.721268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=122ce77e87bf437bbab55556fbf3b73e, user_id=ba2f4fdae3f54f2f841077b1789fa530, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:50.724225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:50.738900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1080/2193] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:50 2022] PUT /identity/v3/projects/122ce77e87bf437bbab55556fbf3b73e/users/ba2f4fdae3f54f2f841077b1789fa530/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:50.750561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7cb5-00b6-4522-8ff8-df4abbdaabce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:50.751099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7cb5-00b6-4522-8ff8-df4abbdaabce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:50.751222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7cb5-00b6-4522-8ff8-df4abbdaabce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:50.783956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-868d7cb5-00b6-4522-8ff8-df4abbdaabce None None] MFA Rules not processed for user `ba2f4fdae3f54f2f841077b1789fa530`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:50.785367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-868d7cb5-00b6-4522-8ff8-df4abbdaabce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:50.916639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1114/2194] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:50.928039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:50.929802 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.003976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159443-9a9b-4ed1-8b93-aa8d6dc53b53 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:51.008543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:51.009611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.009903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.010186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.102509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:51.110133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:51.121574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1081/2195] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:51.462302 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb754b-e301-4dd2-a495-aaa1396bf14d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.462746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb754b-e301-4dd2-a495-aaa1396bf14d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.463088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb754b-e301-4dd2-a495-aaa1396bf14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.499108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-50cb754b-e301-4dd2-a495-aaa1396bf14d None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:51.500509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-50cb754b-e301-4dd2-a495-aaa1396bf14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.505516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:51.570729 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1115/2196] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:33:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:51.593634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:51.604494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:51.668566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a78b4-68e3-4503-8b92-b93fd0eb3170 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:51.674600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:51.675278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.675566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.675839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.685539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:51.687152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:51.695607 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1082/2197] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:33:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:33:51.910726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a81b-cae1-422d-88be-7d69fecd75b2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.910726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a81b-cae1-422d-88be-7d69fecd75b2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.910726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79a81b-cae1-422d-88be-7d69fecd75b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.927140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3a79a81b-cae1-422d-88be-7d69fecd75b2 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:51.928508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3a79a81b-cae1-422d-88be-7d69fecd75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:51.942398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1116/2198] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:33:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:51.955838 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:51.969850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c269f-3827-4a24-82e8-4907b425bf03 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:51.977470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:51.978226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:51.978592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:51.978937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:51.989643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:51.991118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:52.001704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1083/2199] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:33:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:33:52.385469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:52.385759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:52.386044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:52.403984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:52.405851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:52.420228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1117/2200] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:33:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:52.433595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:52.445147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6cadb-d324-4d21-a0e4-fc64427d3a9a service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:52.450722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:52.451494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:52.451777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:52.452052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:52.463052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:52.464831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:52.474566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1084/2201] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:33:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:33:53.700943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:53.703092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:53.767886 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec0cbd-70b1-4776-92b8-911caed3f27e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:53.773223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:53.774141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:53.774434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:53.774713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] PATH_INFO: `/v3/users/1b0cfc8c3a8d41bdada3cb9888648e64` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:53.776261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0cfc8c3a8d41bdada3cb9888648e64)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:53.778495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:53.824524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:53.831656 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0cfc8c3a8d41bdada3cb9888648e64'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.835937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abaf1-834d-4348-b206-5f2e11af54af service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:53.840822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0cfc8c3a8d41bdada3cb9888648e64'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.841676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:53.843088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:53.843373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:53.843651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] PATH_INFO: `/v3/users/adddeb29f55e43789fe92ef73800caa2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:53.845038 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] RBAC: Authorizing `identity:delete_user(user_id=adddeb29f55e43789fe92ef73800caa2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:53.847013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:53.854674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0cfc8c3a8d41bdada3cb9888648e64'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.855324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0cfc8c3a8d41bdada3cb9888648e64'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.864661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-32268590-3b22-4a34-9ab1-d87934d858c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0cfc8c3a8d41bdada3cb9888648e64'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.871334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1118/2202] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:53 2022] DELETE /identity/v3/users/1b0cfc8c3a8d41bdada3cb9888648e64 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:53.875641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:53.875959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:53.876271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:53.894556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:53.896095 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:53.899445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adddeb29f55e43789fe92ef73800caa2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.908611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adddeb29f55e43789fe92ef73800caa2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.909239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adddeb29f55e43789fe92ef73800caa2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.911686 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:53.923698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adddeb29f55e43789fe92ef73800caa2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.931331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-becc829d-9fe2-4194-869e-fa487b1a87be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adddeb29f55e43789fe92ef73800caa2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:53.941175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1085/2203] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:53 2022] DELETE /identity/v3/users/adddeb29f55e43789fe92ef73800caa2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:53.945899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:53.946233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:53.946521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:53.966799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1119/2204] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:53.968402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:53.969687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:53.977211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:53.978607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.014925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:54.039000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1086/2205] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:54.049733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.051436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6128e2a-5a85-4765-9016-b0e312eea305 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.105912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.106264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.106548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.106824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.117089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.118028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc16669-1666-4024-ab67-225540e349f8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.118667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.121800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.122480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.122928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.123278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.130150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1120/2206] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:54.133506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.134989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.145774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1087/2207] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:54.535206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.536803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.551041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028798-1dc7-4cfb-b23c-0593faa79f01 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.556231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.557093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.561414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.562183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.562474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] PATH_INFO: `/v3/projects/0163d466aa594f58a480d7c362f3caa8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.563669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.563963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0163d466aa594f58a480d7c362f3caa8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.566202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.582342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb78b7-fbab-4f93-b35a-fc6fdd6d8508 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.590240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.591578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.592611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.593021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] PATH_INFO: `/v3/projects/9237174420004af8add8d53ec8a45aad` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.594621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9237174420004af8add8d53ec8a45aad)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.597035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.654741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0163d466aa594f58a480d7c362f3caa8', 'request_id': 'req-1795fca4-036c-4ed7-b191-8af5f0c1acac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.667547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0163d466aa594f58a480d7c362f3caa8', 'request_id': 'req-1795fca4-036c-4ed7-b191-8af5f0c1acac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.681642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0163d466aa594f58a480d7c362f3caa8', 'request_id': 'req-1795fca4-036c-4ed7-b191-8af5f0c1acac'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.697609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9237174420004af8add8d53ec8a45aad', 'request_id': 'req-b8d1ed98-128e-4121-9de8-a9bdc027e863'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.706936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9237174420004af8add8d53ec8a45aad', 'request_id': 'req-b8d1ed98-128e-4121-9de8-a9bdc027e863'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.708267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] The token cache is being invalidate because project 0163d466aa594f58a480d7c362f3caa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:54.708862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.709394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.711010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1121/2208] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:54 2022] DELETE /identity/v3/projects/0163d466aa594f58a480d7c362f3caa8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:54.711934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.713263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.716728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9237174420004af8add8d53ec8a45aad', 'request_id': 'req-b8d1ed98-128e-4121-9de8-a9bdc027e863'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.718988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:54.730861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] The token cache is being invalidate because project 9237174420004af8add8d53ec8a45aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:54.731153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.731798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.734311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1088/2209] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:54 2022] DELETE /identity/v3/projects/9237174420004af8add8d53ec8a45aad => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:54.738212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.739557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.796363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fca4-036c-4ed7-b191-8af5f0c1acac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.823524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1ed98-128e-4121-9de8-a9bdc027e863 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:54.830503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.831385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.832012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.832012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] PATH_INFO: `/v3/users/5e3ec13327dc48b2ace67e1819b50891` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.833143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3ec13327dc48b2ace67e1819b50891)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.835112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.850015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:54.863968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:54.864777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.865057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.865344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] PATH_INFO: `/v3/users/ab229da11aac429698ad11d65aa58485` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:54.866740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab229da11aac429698ad11d65aa58485)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:54.869048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:54.899000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3ec13327dc48b2ace67e1819b50891'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.907826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3ec13327dc48b2ace67e1819b50891'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.908637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3ec13327dc48b2ace67e1819b50891'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.923778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3ec13327dc48b2ace67e1819b50891'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.930418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab229da11aac429698ad11d65aa58485'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.931634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3ec13327dc48b2ace67e1819b50891'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.938993 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1089/2210] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:54 2022] DELETE /identity/v3/users/5e3ec13327dc48b2ace67e1819b50891 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:54.939795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:54.940585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab229da11aac429698ad11d65aa58485'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.941282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:54.957780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab229da11aac429698ad11d65aa58485'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.958696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab229da11aac429698ad11d65aa58485'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.968535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-44c468c0-0562-4fa7-a0d9-b8d2f0acad80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab229da11aac429698ad11d65aa58485'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:54.985550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1122/2211] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:54 2022] DELETE /identity/v3/users/ab229da11aac429698ad11d65aa58485 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:54.989809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:54.990227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:54.990474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.011538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:55.012608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.043597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:55.054280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b784eb-f20b-4379-9627-76beba4422a7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:55.099612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1123/2212] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:55.113035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:55.116188 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.142159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:55.143052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.143338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.143618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] PATH_INFO: `/v3/users/ba2f4fdae3f54f2f841077b1789fa530` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.144859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2f4fdae3f54f2f841077b1789fa530)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:55.147612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:55.181243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89456782-f097-48bf-94c1-ce89fab51f33 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:55.185808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:55.186548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.186840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.187084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.197855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:55.199413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:55.202171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:55.210798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2f4fdae3f54f2f841077b1789fa530'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.213846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1124/2213] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:33:55.220638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2f4fdae3f54f2f841077b1789fa530'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.221368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2f4fdae3f54f2f841077b1789fa530'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.234002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2f4fdae3f54f2f841077b1789fa530'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.240676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2f4fdae3f54f2f841077b1789fa530'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.246876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1090/2214] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:54 2022] DELETE /identity/v3/users/ba2f4fdae3f54f2f841077b1789fa530 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:55.520648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:55.522145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.538200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05043215-d429-47da-943c-0387d9ed3e49 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:55.630509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:55.631369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.631707 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.632144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] PATH_INFO: `/v3/projects/8bd538a356c147629432c2e3cf713b10` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.633413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd538a356c147629432c2e3cf713b10)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:55.635405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:55.639943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:55.641958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.656854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a28512-21e4-4191-a6c2-48df7d851611 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:55.662826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:55.663848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.663962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.664239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] PATH_INFO: `/v3/projects/122ce77e87bf437bbab55556fbf3b73e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.665479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=122ce77e87bf437bbab55556fbf3b73e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:55.667452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:55.714946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd538a356c147629432c2e3cf713b10', 'request_id': 'req-d3b50f70-274f-4625-9f29-84e678702d8e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.736925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd538a356c147629432c2e3cf713b10', 'request_id': 'req-d3b50f70-274f-4625-9f29-84e678702d8e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.746556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd538a356c147629432c2e3cf713b10', 'request_id': 'req-d3b50f70-274f-4625-9f29-84e678702d8e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.764511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] The token cache is being invalidate because project 8bd538a356c147629432c2e3cf713b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:55.765065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.766248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.767905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1125/2215] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:55 2022] DELETE /identity/v3/projects/8bd538a356c147629432c2e3cf713b10 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:55.768743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:55.770201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.774526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122ce77e87bf437bbab55556fbf3b73e', 'request_id': 'req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.775267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:55.794390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122ce77e87bf437bbab55556fbf3b73e', 'request_id': 'req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.803395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122ce77e87bf437bbab55556fbf3b73e', 'request_id': 'req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.818369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] The token cache is being invalidate because project 122ce77e87bf437bbab55556fbf3b73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:55.818670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.819471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-49647742-c4d5-4bd6-87c3-3d9553bbcbd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:55.821031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1091/2216] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:55 2022] DELETE /identity/v3/projects/122ce77e87bf437bbab55556fbf3b73e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:55.822415 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374009-a8be-4369-acb7-27a29ff943ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.822703 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374009-a8be-4369-acb7-27a29ff943ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.822985 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374009-a8be-4369-acb7-27a29ff943ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.848623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3374009-a8be-4369-acb7-27a29ff943ab None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:55.850036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3374009-a8be-4369-acb7-27a29ff943ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:55.851365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50f70-274f-4625-9f29-84e678702d8e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:55.925603 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:55.926654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.927150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.927521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] PATH_INFO: `/v3/projects/553646e8f0884b22ba3e3d5c6b173d1b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.929510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553646e8f0884b22ba3e3d5c6b173d1b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:55.937457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:55.959259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1092/2217] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:55.961506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b6bb-eee6-4cf3-90f5-2bd4697f218c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:55.962107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b6bb-eee6-4cf3-90f5-2bd4697f218c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:55.962559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b6bb-eee6-4cf3-90f5-2bd4697f218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:55.992663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9591b6bb-eee6-4cf3-90f5-2bd4697f218c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:55.994387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9591b6bb-eee6-4cf3-90f5-2bd4697f218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.025688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553646e8f0884b22ba3e3d5c6b173d1b', 'request_id': 'req-6e47d3f3-368b-4dad-af7b-b5a5c0263894'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:56.038752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553646e8f0884b22ba3e3d5c6b173d1b', 'request_id': 'req-6e47d3f3-368b-4dad-af7b-b5a5c0263894'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:56.040502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:56.048937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553646e8f0884b22ba3e3d5c6b173d1b', 'request_id': 'req-6e47d3f3-368b-4dad-af7b-b5a5c0263894'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:56.071002 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] The token cache is being invalidate because project 553646e8f0884b22ba3e3d5c6b173d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:56.071310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:56.072467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6e47d3f3-368b-4dad-af7b-b5a5c0263894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:56.074681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1126/2218] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:55 2022] DELETE /identity/v3/projects/553646e8f0884b22ba3e3d5c6b173d1b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:56.079500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32517a3-5435-4fdc-9235-5a517cb7dcff None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.081314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32517a3-5435-4fdc-9235-5a517cb7dcff None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.081314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32517a3-5435-4fdc-9235-5a517cb7dcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.096925 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1093/2219] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.098255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.098643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.099024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.112141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d32517a3-5435-4fdc-9235-5a517cb7dcff None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:56.113712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d32517a3-5435-4fdc-9235-5a517cb7dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.124730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:56.126147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.129411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1127/2220] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.131376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005a80-b406-4f47-8dfe-046eba1c2278 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.131376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005a80-b406-4f47-8dfe-046eba1c2278 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.131826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005a80-b406-4f47-8dfe-046eba1c2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.141263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1094/2221] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.142488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.153585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513db00c-2af9-440c-98dc-4fe362333070 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.158803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.159583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.159984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.160312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.161116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:56.161538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.163278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.165061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4d005a80-b406-4f47-8dfe-046eba1c2278 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:56.166469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d005a80-b406-4f47-8dfe-046eba1c2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.174951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1128/2222] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.176244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.176524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.176802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.185380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1095/2223] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:56.190204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.198067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:56.199537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.204066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8789b-c031-430a-ad08-ab600f4bc676 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.208831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1129/2224] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.212172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.213373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.214208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.214648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.215987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.215987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:56.215987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.217617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.224082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea4482-c0d0-418a-b375-46c179482c74 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.230571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.231237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.231521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.231948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.232645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.236552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1096/2225] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:56.236552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.236745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.246486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5304bf-60d3-4ba5-b855-e7ab1175ad8e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.257966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.258648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.258940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.259229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.259967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:56.260357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.261908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.267367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:56.281086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1097/2226] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:56.281571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1130/2227] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.282616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.282730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.295248 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bafe65-c524-4259-bf02-ebfd0f9c5de0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.359933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.360770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.361153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.361631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.362391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.363978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2fa29-79df-40ad-ba50-a2f2c73b34b4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.364509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.369977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.370743 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.371096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.371447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.372300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.374591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.400718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1131/2228] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.400718 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.409487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabee39-ef73-4a81-9500-6fa905afd060 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.411605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1098/2229] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.422108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.474106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.475244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.475559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.475962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.479725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.484999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.497839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380720-02aa-4181-96cf-5e70c68a5a87 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.509761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.512290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.512880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.513706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.514610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.516303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.528192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1099/2230] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:56.529568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1132/2231] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.533800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.534518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.548058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864b227-f55d-4531-834a-1bf6959130f7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.566035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:56.635470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.636552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.637172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.637624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.638623 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.641238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.641592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242438-c9d7-4691-995d-f4fcf06a673d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.647858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.648657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.649044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.649902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.650476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.654080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.678611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1133/2232] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:56.679702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.680096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1100/2233] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.681212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.693786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b6c11-7fd1-476b-adcd-a9535cd066db admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.694740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fce66-4768-4d54-9053-205c4c96f9ce admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.699310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.700373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.700678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.701062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.701515 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] PATH_INFO: `/v3/projects/285799be947b4df3ba0f111aa2aa44d0/users/9b5adeffa0d84d0c9595d557a3103b99/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.701755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.702194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.702612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.704331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.707554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.716467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:56.720913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] RBAC: Authorizing `identity:create_grant(project_id=285799be947b4df3ba0f111aa2aa44d0, user_id=9b5adeffa0d84d0c9595d557a3103b99, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.724378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.737917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1134/2234] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/users => generated 300 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:56.738741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.744820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1101/2235] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:56 2022] PUT /identity/v3/projects/285799be947b4df3ba0f111aa2aa44d0/users/9b5adeffa0d84d0c9595d557a3103b99/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:56.746978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.759311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02790e-7cf6-414d-944d-b2972320ce8c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.845765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.846734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.847084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.847847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.848877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.853233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8733db5-3d00-401d-98bf-345eaa91ea16 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.873932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1135/2236] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:56.873932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.873932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.873932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.894392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b3b67-68be-4acc-a971-0fd0a1d1585e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.900624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.902340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.902790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.903087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.904050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.905256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] MFA Rules not processed for user `9b5adeffa0d84d0c9595d557a3103b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:56.906581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:56.906968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.919359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1102/2237] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:56.922418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.935480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec3bf8-41c7-41d2-86c1-1168a12b9522 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.941876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.942881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.943384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.943672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.944752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.946253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:56.958008 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1103/2238] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:56.962318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.975436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf825-40f6-4a9c-818e-4b54dd2fe7f3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:56.984896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1136/2239] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:56.985981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:56.991727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:56.992495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:56.993266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:56.993688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:56.994428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:56.996753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.003235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ca3a-efdd-4893-9c8a-6fc836b71973 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.009463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.010980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1104/2240] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:57.011930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.013404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.014471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.014689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] PATH_INFO: `/v3/projects/5eef59a370e74eaa869576855cc476f2/users/b63f00faf3054e34afbaebe31becfd7a/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.026786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46848ec-48c0-4379-948a-5e49c148f916 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.037894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.038631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.038917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.039228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.039970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.042058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.047954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef59a370e74eaa869576855cc476f2, user_id=b63f00faf3054e34afbaebe31becfd7a, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.051420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.072203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1137/2241] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:56 2022] PUT /identity/v3/projects/5eef59a370e74eaa869576855cc476f2/users/b63f00faf3054e34afbaebe31becfd7a/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:57.073473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.077422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1105/2242] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:57 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:33:57.079002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.135648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:57.158246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:57.172524 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702678-d665-45f1-a363-d4e27e8df693 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.174477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47eddc9-e83a-450e-8fd9-6b1d5dacdf51 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.177756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.179362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.179499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.179760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.180113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.180402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.180541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] PATH_INFO: `/v3/projects/bff40795cf964221b19decd2be3aa283/users/34bfad6619eb4a798b188cd1ef9ed3b0/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.180678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.181390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.183728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2509f-50f4-424d-8f79-804aa74e7e50 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.197513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff40795cf964221b19decd2be3aa283, user_id=34bfad6619eb4a798b188cd1ef9ed3b0, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.200157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.210463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1106/2243] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:57 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:57.215036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d8de-7c0e-4270-9e22-464eed92953b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.215501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d8de-7c0e-4270-9e22-464eed92953b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.215994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9d8de-7c0e-4270-9e22-464eed92953b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.216454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1138/2244] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:57 2022] PUT /identity/v3/projects/bff40795cf964221b19decd2be3aa283/users/34bfad6619eb4a798b188cd1ef9ed3b0/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:57.217175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.259957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0ac9d8de-7c0e-4270-9e22-464eed92953b None None] MFA Rules not processed for user `b63f00faf3054e34afbaebe31becfd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:57.261251 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0ac9d8de-7c0e-4270-9e22-464eed92953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:57.320583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9d59f-782a-4c01-be25-40f0e68878ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.327538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.328198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.328484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.333560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.333560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.333560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.336383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1107/2245] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:57.337815 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.338174 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.338512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.349484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1139/2246] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:57.353531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.365996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62767ea-0985-424a-8555-f854ab4a737e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.367193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] MFA Rules not processed for user `34bfad6619eb4a798b188cd1ef9ed3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:57.368528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:57.370841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.371732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.372020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.372428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] PATH_INFO: `/v3/projects/880025bc84ae4718b7a4c69606a0d0a3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.389640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880025bc84ae4718b7a4c69606a0d0a3, user_id=5f9ee80ab3ff4624bfb84d4e021d6c5f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.392023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.411536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1140/2247] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:57 2022] PUT /identity/v3/projects/880025bc84ae4718b7a4c69606a0d0a3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:57.411536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.414237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:57.472856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1108/2248] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:57.474559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.494308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552ef02-4692-4c85-b95e-66f44ff0814e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.517154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f1f14-4d1b-4a41-aefe-a3d2d2e72546 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.521256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.522141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.522303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.522583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.565593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:57.591674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:57.601620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.602309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.602601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.602884 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.603629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.605463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.610604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.612337 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.619888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1109/2249] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:33:57.619888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.621600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:57.632706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1141/2250] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:57.633623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.647246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98568383-fc73-466c-b309-624b61645cf0 service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.652278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.653791 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.654083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.654907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] PATH_INFO: `/v3/projects/880025bc84ae4718b7a4c69606a0d0a3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.657653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=880025bc84ae4718b7a4c69606a0d0a3, user_id=5f9ee80ab3ff4624bfb84d4e021d6c5f, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.660575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28bded-08ee-4edb-9e2c-96f3e13af6fc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.676955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1142/2251] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:57 2022] PUT /identity/v3/projects/880025bc84ae4718b7a4c69606a0d0a3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:57.688043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.688731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.689074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.721024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a65bc-a41b-4d1d-98c7-e33c75cd3fed admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.724419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] MFA Rules not processed for user `5f9ee80ab3ff4624bfb84d4e021d6c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:57.725854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:57.791178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.792563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.792924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.794426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.817833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1143/2252] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:57.878625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.880296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:57.901910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1110/2253] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:57.948447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:57.960087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b0f87-bd6e-4269-bdd0-6647605ee00e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:57.965314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:57.966281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:57.966567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:57.967434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:57.988483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:57.990327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:58.005529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1144/2254] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:58.784584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:58.786524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:58.797319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:58.893536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:58.908190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d9d13-fcff-4ec5-91df-c0a67779208b service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:58.914442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:58.915345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:58.915627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:58.915903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] PATH_INFO: `/v3/users/9b5adeffa0d84d0c9595d557a3103b99` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:58.917293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5adeffa0d84d0c9595d557a3103b99)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:58.919344 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:58.925173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088185-2037-4067-893c-a9711a55f36b service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:58.929877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:58.930839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:58.931201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:58.931484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] PATH_INFO: `/v3/users/d1ac1cdd046a4ae4b9f41b75d347d997` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:58.933239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ac1cdd046a4ae4b9f41b75d347d997)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:58.937158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:58.973797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:58.975423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5adeffa0d84d0c9595d557a3103b99'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.984411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5adeffa0d84d0c9595d557a3103b99'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.991760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ac1cdd046a4ae4b9f41b75d347d997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.998565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5adeffa0d84d0c9595d557a3103b99'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.999218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ac1cdd046a4ae4b9f41b75d347d997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.999454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5adeffa0d84d0c9595d557a3103b99'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:58.999891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ac1cdd046a4ae4b9f41b75d347d997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.006726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-73f96779-fd20-4692-a0d7-ba94bf4e4510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5adeffa0d84d0c9595d557a3103b99'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.012578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ac1cdd046a4ae4b9f41b75d347d997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.012730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1145/2255] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:58 2022] DELETE /identity/v3/users/9b5adeffa0d84d0c9595d557a3103b99 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:59.017598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.017975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.018322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.019679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ac1cdd046a4ae4b9f41b75d347d997'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.026172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1111/2256] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:58 2022] DELETE /identity/v3/users/d1ac1cdd046a4ae4b9f41b75d347d997 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:59.037616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:59.038873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.098720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1146/2257] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:59.109057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.110714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.179971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a3744-d27c-4f4f-b0bb-be2c9e170e5f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.183936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.184582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.184864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.185141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.195588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.197283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149c409-1930-4d67-860d-392b64536dc9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.207888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1112/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:33:59.214437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.220980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.241613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7a9b1-129b-4024-8c18-d8d954a14fe1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.249149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.250165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.250455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.250761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] PATH_INFO: `/v3/projects/bd2e10a0973e431db6cc11cc924fd826` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.252201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2e10a0973e431db6cc11cc924fd826)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.255267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.315367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2e10a0973e431db6cc11cc924fd826', 'request_id': 'req-9035815f-8421-4038-bfb8-a56b16757ad3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.323141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2e10a0973e431db6cc11cc924fd826', 'request_id': 'req-9035815f-8421-4038-bfb8-a56b16757ad3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.332816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2e10a0973e431db6cc11cc924fd826', 'request_id': 'req-9035815f-8421-4038-bfb8-a56b16757ad3'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.349607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] The token cache is being invalidate because project bd2e10a0973e431db6cc11cc924fd826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:59.349607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.349607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9035815f-8421-4038-bfb8-a56b16757ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.349607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1147/2259] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:59 2022] DELETE /identity/v3/projects/bd2e10a0973e431db6cc11cc924fd826 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:59.355112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.355409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.355694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.375481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:59.376709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.448170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1113/2260] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:33:59.456710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.456968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.457259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.476862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:33:59.478370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.487430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1148/2261] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:33:59.493752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.506153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8f863-2b76-4010-a4c2-56f272d8d6f4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.513526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.514861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.514861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.514861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.517619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:33:59.517619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.517619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.524569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:59.534160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1114/2262] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:33:59.535379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.542546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.544677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.546533 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd28c-9a75-4233-90de-e135db9511d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.558828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df302baf-c1c0-42c5-b23c-31640ae8cb7f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.563951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.564925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.565256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.565587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.565775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.565882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] PATH_INFO: `/v3/projects/285799be947b4df3ba0f111aa2aa44d0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.566200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.566605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.567316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.567775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=285799be947b4df3ba0f111aa2aa44d0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.570291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.570777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.606275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1149/2263] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:59.610446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.630647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:59.634440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285799be947b4df3ba0f111aa2aa44d0', 'request_id': 'req-2552f5c4-9a66-486f-b5e8-2947d45fefa6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.643250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285799be947b4df3ba0f111aa2aa44d0', 'request_id': 'req-2552f5c4-9a66-486f-b5e8-2947d45fefa6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.650820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285799be947b4df3ba0f111aa2aa44d0', 'request_id': 'req-2552f5c4-9a66-486f-b5e8-2947d45fefa6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.662954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] The token cache is being invalidate because project 285799be947b4df3ba0f111aa2aa44d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:33:59.663254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.663921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.665752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1115/2264] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:33:59 2022] DELETE /identity/v3/projects/285799be947b4df3ba0f111aa2aa44d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:59.669241 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.671100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.679562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cdd90-68ff-4d89-ba95-09046e3771ab admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.743428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.744035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.744316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.744590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.745271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.746106 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f5c4-9a66-486f-b5e8-2947d45fefa6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.747220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.750799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.751657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.751941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.752219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] PATH_INFO: `/v3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.753442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9ee80ab3ff4624bfb84d4e021d6c5f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.755354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.771950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1150/2265] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:33:59.775847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.778106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.790633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a3b56-1d32-4f0c-886e-b68fd6a5e640 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.796284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.796963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.797248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.797565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.798296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.799938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.810822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1151/2266] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:59.814695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.817264 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9ee80ab3ff4624bfb84d4e021d6c5f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.824349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2fc3-6ddb-4b8e-9c3e-913973e4d332 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.825207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9ee80ab3ff4624bfb84d4e021d6c5f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.825952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9ee80ab3ff4624bfb84d4e021d6c5f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.829508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.830300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.830627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.830909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.831783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.833404 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.839006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9ee80ab3ff4624bfb84d4e021d6c5f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.846643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1152/2267] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:33:59.850022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0165bea8-5194-4763-b1df-0ca04b60dd4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9ee80ab3ff4624bfb84d4e021d6c5f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:33:59.852137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:33:59.852558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:33:59.852699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:33:59.860699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1116/2268] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:33:59 2022] DELETE /identity/v3/users/5f9ee80ab3ff4624bfb84d4e021d6c5f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:33:59.866970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312430-2baa-42d9-8eb0-f7db596b82c5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:33:59.940385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:33:59.941815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.942106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.942408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] PATH_INFO: `/v3/projects/926f1338d73b48aebc0431e2121a0dba/users/a48066ed22e14c64a316fe4d66394f44/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:33:59.958690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926f1338d73b48aebc0431e2121a0dba, user_id=a48066ed22e14c64a316fe4d66394f44, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:33:59.961358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:33:59.977933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1153/2269] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:33:59 2022] PUT /identity/v3/projects/926f1338d73b48aebc0431e2121a0dba/users/a48066ed22e14c64a316fe4d66394f44/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:33:59.983387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:33:59.983861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:33:59.983954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.010479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:00.011715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.121519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1117/2270] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:00.135269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.139396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.175806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:00.239678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.241176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.254404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e856fb0-bcd3-4662-845a-37b4bd111ec5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.259064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678125-8eb3-4285-b31f-2ad748fddc02 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.259318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.260191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.260472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.260747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] PATH_INFO: `/v3/projects/880025bc84ae4718b7a4c69606a0d0a3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.261946 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880025bc84ae4718b7a4c69606a0d0a3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.264383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.265346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.266265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.266626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.267289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.280705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.282562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e8894-dbd2-4dc9-8489-00a0ebf72e81 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.298887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1154/2271] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:00.327009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880025bc84ae4718b7a4c69606a0d0a3', 'request_id': 'req-1e6f1710-1389-4474-80d8-0a02974b1ad4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:00.335499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880025bc84ae4718b7a4c69606a0d0a3', 'request_id': 'req-1e6f1710-1389-4474-80d8-0a02974b1ad4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:00.341322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880025bc84ae4718b7a4c69606a0d0a3', 'request_id': 'req-1e6f1710-1389-4474-80d8-0a02974b1ad4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:00.358518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] The token cache is being invalidate because project 880025bc84ae4718b7a4c69606a0d0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:00.358820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:00.359485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e6f1710-1389-4474-80d8-0a02974b1ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:00.361506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1118/2272] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:00 2022] DELETE /identity/v3/projects/880025bc84ae4718b7a4c69606a0d0a3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:34:00.366238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.366528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.366809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.389290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:00.390648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.470627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1155/2273] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:00.478542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.478834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.479109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.500786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:00.502420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.510785 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1119/2274] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:00.515056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.527264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca1ff0-b28a-40c0-9d52-26a5e37aa687 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.536417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.536417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.536417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.536417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.536864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:34:00.536864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.537583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.555592 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1156/2275] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:34:00.556855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.567626 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf3294-b356-4ab8-83c9-da9d9e2585aa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.572365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.573034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.573320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.573631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.574328 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.576788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.605366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1120/2276] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:00 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:00.610175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.620836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617c8c7-1d0a-4ae4-b97c-31240d21bc1f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.685659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.686361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.686648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.686929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.687658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.689726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.718593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.719285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1157/2277] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:00 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:00.730387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623585e6-f67d-4b81-a182-4185e9552a75 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.737071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.738031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.738322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.738651 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.739407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.741069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.752171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1121/2278] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:00.756253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.766624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1c5e1-4271-4fd1-bad5-069ed11cc0eb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.771437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.772103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.772394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.772752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.773561 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.775133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cadd02-1b26-43be-8d60-b567a3c6bb3b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.788137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1158/2279] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:00.791962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:00.804474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d563-75df-437b-871e-ecea6c73c976 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:00.812342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:00.813118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.813477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.813757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] PATH_INFO: `/v3/projects/1b3a607b2e0346f2b7393c4a6ce31457/users/0cd5d671ef6b4672a5624534470d3aed/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.840186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b3a607b2e0346f2b7393c4a6ce31457, user_id=0cd5d671ef6b4672a5624534470d3aed, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:00.845581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea4809-e033-4b7d-8700-4b4a17d65320 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:00.859804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1122/2280] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:00 2022] PUT /identity/v3/projects/1b3a607b2e0346f2b7393c4a6ce31457/users/0cd5d671ef6b4672a5624534470d3aed/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:00.870174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffee4d-fd2b-492c-902e-954567b95a45 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:00.870992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffee4d-fd2b-492c-902e-954567b95a45 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:00.871471 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffee4d-fd2b-492c-902e-954567b95a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:00.903967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8fffee4d-fd2b-492c-902e-954567b95a45 None None] MFA Rules not processed for user `0cd5d671ef6b4672a5624534470d3aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:00.906565 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8fffee4d-fd2b-492c-902e-954567b95a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:00.918807 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:00.992671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1159/2281] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:04.660973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:04.661666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:04.661666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:04.689572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] MFA Rules not processed for user `a48066ed22e14c64a316fe4d66394f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:04.690931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:04.699410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:04.769056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1123/2282] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:04.778792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:04.779407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:04.779407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:04.798503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] MFA Rules not processed for user `a48066ed22e14c64a316fe4d66394f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:04.799771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:04.807335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1160/2283] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:04.821179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:04.822866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:04.907445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f7557-d605-4525-a3f6-f22386f4983c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:04.918662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:04.919142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:04.919340 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:04.919617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:04.930886 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:04.934295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de6986-210b-4b72-9e9e-b6ea5dc2a07e service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:04.944336 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1124/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:06.667056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:06.668528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:06.747020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211e719-807e-4d9e-b8f4-eb014892212c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:06.761498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:06.762501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:06.762798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:06.763079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] PATH_INFO: `/v3/users/34bfad6619eb4a798b188cd1ef9ed3b0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:06.765322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bfad6619eb4a798b188cd1ef9ed3b0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:06.767731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:06.818276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bfad6619eb4a798b188cd1ef9ed3b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:06.832264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bfad6619eb4a798b188cd1ef9ed3b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:06.845206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bfad6619eb4a798b188cd1ef9ed3b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:06.846048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bfad6619eb4a798b188cd1ef9ed3b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:06.852734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bfad6619eb4a798b188cd1ef9ed3b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:06.859366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1161/2285] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:34:06 2022] DELETE /identity/v3/users/34bfad6619eb4a798b188cd1ef9ed3b0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:06.863628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:06.864122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:06.864339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:06.883133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:06.884636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:06.957213 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1125/2286] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:06.967020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:06.968675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.001526 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:07.040118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8effa-9ed5-448c-806a-6afc39be96c3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:07.043895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:07.044551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.044859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.045124 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.054956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:07.056423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:07.066508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1162/2287] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:07.326071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:07.327780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.374905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:07.386626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:07.388742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.420470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60d81-2e37-4785-8ad3-04aeade3f03e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:07.424307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:07.425046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.425458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.425812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.427991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.441170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a997a1-4413-4ae4-90cb-247310d883cb service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:07.451496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:07.453047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.456051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.456843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] PATH_INFO: `/v3/projects/bff40795cf964221b19decd2be3aa283` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.458989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff40795cf964221b19decd2be3aa283)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:07.461187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:07.522161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:07.524062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59664c51-a6a2-4340-ace8-6789bb80cfc3 service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:07.531706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff40795cf964221b19decd2be3aa283', 'request_id': 'req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:07.542958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff40795cf964221b19decd2be3aa283', 'request_id': 'req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:07.543981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1126/2288] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:34:07.550195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff40795cf964221b19decd2be3aa283', 'request_id': 'req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:07.565814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] The token cache is being invalidate because project bff40795cf964221b19decd2be3aa283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:07.566225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:07.567564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7cbe6d5-f6b5-45c3-bb64-09d93a868d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:07.574901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1163/2289] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:07 2022] DELETE /identity/v3/projects/bff40795cf964221b19decd2be3aa283 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:34:07.575913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.576263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.576530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.612588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:07.614588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.764410 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1127/2290] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:07.772907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.773462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.773462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.810666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:07.811456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:07.823235 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1164/2291] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:07.824293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:07.842682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64149d90-4eb5-49c8-b246-629d3972eec2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:07.848300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:07.849086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.849556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.850072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.850823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:34:07.851545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:07.853226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:07.879525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1128/2292] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:34:07.881148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:07.891546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d39c9-99a6-47c8-b2e8-f85eb0210507 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:07.897137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:07.897551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:07.897836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:07.898115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:07.898867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:07.900956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:07.937611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1165/2293] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:07 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:07.938679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:07.951998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6732cc-8621-4b8d-861c-bf58d5934c15 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:08.142043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:08.143943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.144234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.144518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.148418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:08.152496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:08.219414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1129/2294] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:07 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:08.225312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:08.246690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ec639-323e-4cd6-baa8-34baa8a20ea6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:08.266225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:08.267153 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.267452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.267736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.268481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:08.270148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:08.290342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1166/2295] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:08.295092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:08.320026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731876e9-6db9-4616-a3ca-c9f414c0a301 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:08.327878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:08.328656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.329036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.329597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.330394 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:08.332478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67ca5c-88d6-459b-94a4-076c5ab5a3dd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:08.334286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:08.360943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1130/2296] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:08.368411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:08.393778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b87c0-dd99-40d7-b123-9ada651c8b8e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:08.410785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:08.417483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.417483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.417483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] PATH_INFO: `/v3/projects/c82ac9fcca384f0793c21f7e06b234bc/users/7d098bacbe444141b25a14d9d3d06cc1/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.452265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82ac9fcca384f0793c21f7e06b234bc, user_id=7d098bacbe444141b25a14d9d3d06cc1, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:08.454959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:08.502314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1167/2297] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:08 2022] PUT /identity/v3/projects/c82ac9fcca384f0793c21f7e06b234bc/users/7d098bacbe444141b25a14d9d3d06cc1/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:08.503692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23786-d65b-4791-836c-a3dc677d7caf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.503989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23786-d65b-4791-836c-a3dc677d7caf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.504308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23786-d65b-4791-836c-a3dc677d7caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.535143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c5e23786-d65b-4791-836c-a3dc677d7caf None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:08.536561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c5e23786-d65b-4791-836c-a3dc677d7caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:08.636098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1131/2298] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:08.647008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:08.653240 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:08.727183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.727481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.727763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.728326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:08.736319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98f233-cf99-4e3e-ae5a-3b5f8daae625 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:08.743424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:08.744273 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:08.745463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:08.745917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:08.760272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] MFA Rules not processed for user `a48066ed22e14c64a316fe4d66394f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:08.761590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:08.770255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:08.771829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd696bee-734f-4cdd-8478-0ae63bfa7079 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:08.789497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1168/2299] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:08.837999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1132/2300] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:15.934654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:15.935146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:15.938826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:15.980435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] MFA Rules not processed for user `7d098bacbe444141b25a14d9d3d06cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:15.982233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:16.127156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1169/2301] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:16.131186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.155717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1011e5-01dd-400b-ba3f-fe4261267583 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.162365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.164541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.164541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.164541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.165284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.167309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.213777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1133/2302] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:16 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:34:16.218239 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.235802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee95610-e9df-4c2d-b113-c7fe68b22255 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.373562 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.374282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.374563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.375044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.375544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.377558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.408438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1170/2303] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:16 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:34:16.412427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.429818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12ffbb-9dd7-4ac5-a6c9-79c7a6fadd6e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.438062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.438833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.439324 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.439667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.440466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.442171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.454027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1134/2304] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:16.460872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.471275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c7a3f-6476-4712-ba28-e12df2a151f1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.476999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.478436 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.478659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.478940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] PATH_INFO: `/v3/projects/4f591e6efff3443b9561899c037aa719/users/c51cdc6c95bc4bef9db8ea023de08ec8/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.499993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f591e6efff3443b9561899c037aa719, user_id=c51cdc6c95bc4bef9db8ea023de08ec8, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.503271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.525372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1171/2305] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:16 2022] PUT /identity/v3/projects/4f591e6efff3443b9561899c037aa719/users/c51cdc6c95bc4bef9db8ea023de08ec8/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:16.529844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.542358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29240-1d01-4660-a7c6-a766c305902b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.615738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.616412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.616697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.616973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.617739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.619360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.634602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1135/2306] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:16.639031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:16.657228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc3a5-2a20-417b-90f4-1bf594e0fc70 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:16.662589 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:16.663609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:16.663992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:16.664369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] PATH_INFO: `/v3/projects/4f591e6efff3443b9561899c037aa719/users/c51cdc6c95bc4bef9db8ea023de08ec8/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:16.666553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f591e6efff3443b9561899c037aa719, user_id=c51cdc6c95bc4bef9db8ea023de08ec8, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:16.670033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897da62-e01a-4e53-8878-1f75dc507945 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:16.685122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1172/2307] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:16 2022] PUT /identity/v3/projects/4f591e6efff3443b9561899c037aa719/users/c51cdc6c95bc4bef9db8ea023de08ec8/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:19.562811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:19.564686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:19.576656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:19.582764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859383-1f82-42ed-987e-3f48d0848a26 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:19.656814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:19.658134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:19.658823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:19.659331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] PATH_INFO: `/v3/users/777c2f7283914f638dcf6e503ddfd36b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:19.660833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=777c2f7283914f638dcf6e503ddfd36b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:19.664543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:19.728123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777c2f7283914f638dcf6e503ddfd36b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:19.738061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777c2f7283914f638dcf6e503ddfd36b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:19.738801 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777c2f7283914f638dcf6e503ddfd36b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:19.755632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777c2f7283914f638dcf6e503ddfd36b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:19.767820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777c2f7283914f638dcf6e503ddfd36b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:19.780507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1136/2308] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:34:19 2022] DELETE /identity/v3/users/777c2f7283914f638dcf6e503ddfd36b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:19.782318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:19.782872 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:19.783073 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:19.803667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:19.804928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:19.842654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:19.882868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1173/2309] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:19.893192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:19.895096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:19.972932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b7482-4425-45ca-aafd-663ef77b2d2e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:19.976570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:19.977445 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:19.977734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:19.978513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:20.006725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:20.008375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a356dd-7b71-4911-8ced-1ec554a534ec service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:20.030079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1137/2310] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:20.629850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:20.631583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:20.647734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75c106-fbfe-4fd8-9ea2-783e20f0ea53 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:20.655999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:20.656839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:20.657087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:20.657392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] PATH_INFO: `/v3/projects/5eec78cc6e6f43708497e41c53f54fc5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:20.658629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eec78cc6e6f43708497e41c53f54fc5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:20.660871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:20.732825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eec78cc6e6f43708497e41c53f54fc5', 'request_id': 'req-81e21611-00e2-44fc-ba9a-8fd86abf991b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:20.742227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eec78cc6e6f43708497e41c53f54fc5', 'request_id': 'req-81e21611-00e2-44fc-ba9a-8fd86abf991b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:20.748912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eec78cc6e6f43708497e41c53f54fc5', 'request_id': 'req-81e21611-00e2-44fc-ba9a-8fd86abf991b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:20.761368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] The token cache is being invalidate because project 5eec78cc6e6f43708497e41c53f54fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:20.761559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:20.762476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-81e21611-00e2-44fc-ba9a-8fd86abf991b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:20.764362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1174/2311] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:20 2022] DELETE /identity/v3/projects/5eec78cc6e6f43708497e41c53f54fc5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:20.770665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:20.771185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:20.771185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:20.791035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:20.800972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:20.838810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:20.925306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1138/2312] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:20.934637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:20.935848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:20.935848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:20.974808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:20.976070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:20.992618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:20.992874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1175/2313] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:21.005767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defb0c2-c4a8-4e94-bfad-62f0bb2df537 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.012036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.013088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.013566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.013786 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.014861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:34:21.015261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.021980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.033382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1139/2314] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:34:21.036536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.047412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80652a-fe75-46e8-ba8a-711f858c7882 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.052855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.053737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.054138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.054491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.055304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.057397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.094716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1176/2315] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:21 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:21.098893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.110893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9e1c6-ae2a-460a-a9a6-2b439af452d3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.186094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.186809 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.187339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.187339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.188164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.190374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.217833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1140/2316] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:21 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:21.221586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.235649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247acb75-3b14-43c4-ba75-eba71ee57ef0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.240638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.242132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.242422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.242705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.243827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.245709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.265676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1177/2317] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:21.280689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.306822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ca8a-f2fd-4411-8146-119f86f66698 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.315272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.316712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.317063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.317601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.318467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.320386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6af2c9-032c-42f7-9e8e-49285a16ef0b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.338857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1141/2318] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:21.340625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.350832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1605b11-d015-4f2f-b063-458a662ecc7e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.359139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.360314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.360314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.360614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] PATH_INFO: `/v3/projects/aac705367c454623a23f11bfa8d25756/users/42165c1e1b324fedbb926ac20f0f920a/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.380852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac705367c454623a23f11bfa8d25756, user_id=42165c1e1b324fedbb926ac20f0f920a, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.384403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.403570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1178/2319] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:21 2022] PUT /identity/v3/projects/aac705367c454623a23f11bfa8d25756/users/42165c1e1b324fedbb926ac20f0f920a/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:21.409271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b4e0-83e2-406b-97e9-f5c2965de144 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.409504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b4e0-83e2-406b-97e9-f5c2965de144 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.409793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b4e0-83e2-406b-97e9-f5c2965de144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.437259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5314b4e0-83e2-406b-97e9-f5c2965de144 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:21.438561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5314b4e0-83e2-406b-97e9-f5c2965de144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:21.520285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1142/2320] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:21.532069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:21.535006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:21.670879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573b668-1c58-4b17-b4b5-60ec29642cda admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:21.674953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:21.675891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:21.676281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:21.676658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:21.685722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:21.690464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:21.692313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:21.705406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1179/2321] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:22.107358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a90d-7726-4841-903c-4814e666f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:22.108107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a90d-7726-4841-903c-4814e666f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:22.108107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a90d-7726-4841-903c-4814e666f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:22.159815 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d9b0a90d-7726-4841-903c-4814e666f1c3 None None] MFA Rules not processed for user `c51cdc6c95bc4bef9db8ea023de08ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:22.161936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d9b0a90d-7726-4841-903c-4814e666f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:22.200570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:22.265854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1143/2322] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:22.284966 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:22.286783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:22.438213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7289be-2025-4b30-9477-ce8f634db266 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:22.448496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:22.449448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:22.450077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:22.451612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:22.459024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:22.556388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:22.558555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82a68-2d5a-4d21-8cf9-6397773ff7ac service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:22.583655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1180/2323] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:24.080188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:24.080677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:24.081902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:24.122275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] MFA Rules not processed for user `7d098bacbe444141b25a14d9d3d06cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:24.123912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:24.132018 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1144/2324] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:27.970841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:27.971835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:27.972184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.066574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] MFA Rules not processed for user `42165c1e1b324fedbb926ac20f0f920a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:28.068145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:28.342450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.350088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1181/2325] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:28.356151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e0f6-d8f8-4583-8127-e85b001370d4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.363271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.365046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.365635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.366156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.368591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.372983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.425800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1145/2326] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:28 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:34:28.429598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.446246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eabc0f-9c1c-4397-8191-075098abcecd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.470647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:28.523750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.524772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.525737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.526010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.527178 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.529996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.570102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1182/2327] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:28 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:34:28.571341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.583197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a41568-b898-47ca-9013-7a3da5b2ed3b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.588092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.588905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.589089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.589444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.590184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.592305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.609642 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.613485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1146/2328] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:34:28.624847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92794c6a-4922-44c4-97f0-475d7df842cb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.629839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.630799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.631090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.631366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] PATH_INFO: `/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2/users/16d7162fcec44e00a4078735725b124b/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.648091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a90fcd1e654fb09b6d48c1f6c5d6f2, user_id=16d7162fcec44e00a4078735725b124b, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.650833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.670036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1183/2329] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:28 2022] PUT /identity/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2/users/16d7162fcec44e00a4078735725b124b/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:28.678149 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.707746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a98a4d-f0e4-4569-9ca8-0b1f0bb77031 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.820154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.821915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.822356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.822917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.824717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.836777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7d3b-bde5-4d37-be9c-e00670a1b1e1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.857552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1147/2330] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:28.861868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:28.877136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ff14f-c18d-427f-a476-dbab978f2d00 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:28.894771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:28.895921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:28.896091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:28.896484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] PATH_INFO: `/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2/users/16d7162fcec44e00a4078735725b124b/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:28.899149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a90fcd1e654fb09b6d48c1f6c5d6f2, user_id=16d7162fcec44e00a4078735725b124b, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:28.902572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:28.929376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1184/2331] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:28 2022] PUT /identity/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2/users/16d7162fcec44e00a4078735725b124b/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:35.024674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:35.025225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:35.025484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:35.061701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] MFA Rules not processed for user `16d7162fcec44e00a4078735725b124b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:35.063435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:35.163271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:35.173522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1148/2332] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:35.185636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:35.187496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:35.276333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12cba-2a78-4240-b741-620c7fc435a9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:35.282528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:35.283268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:35.283618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:35.283965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:35.351091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:35.352864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fd8ea-d7a5-43fb-98d6-b244a63e16d8 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:35.363340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1185/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:37.968919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:37.970372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:38.044344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91cf95-d27c-4e5b-8931-b1538919458f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:38.049322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:38.050259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:38.050544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:38.050819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] PATH_INFO: `/v3/users/b63f00faf3054e34afbaebe31becfd7a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:38.052053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63f00faf3054e34afbaebe31becfd7a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:38.054234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:38.114996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63f00faf3054e34afbaebe31becfd7a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.123446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63f00faf3054e34afbaebe31becfd7a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.124003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63f00faf3054e34afbaebe31becfd7a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.137027 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63f00faf3054e34afbaebe31becfd7a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.144010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63f00faf3054e34afbaebe31becfd7a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.151207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1149/2334] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:34:37 2022] DELETE /identity/v3/users/b63f00faf3054e34afbaebe31becfd7a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:38.156096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:38.157023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:38.157490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:38.178453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:38.179995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:38.189602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:38.275049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1186/2335] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:38.285294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:38.286891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:38.377864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73498e-42f6-4e17-ac9f-5ae498feaad5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:38.383212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:38.384148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:38.384435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:38.384722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:38.395519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:38.396943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0e1f4-417d-4af3-b54b-d2a16e87c085 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:38.420889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1150/2336] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:38.758294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:38.760114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:38.796170 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1e831-b3da-4472-b31e-0775bc8bcb4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:38.802617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:38.803806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:38.804259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:38.804989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] PATH_INFO: `/v3/projects/5eef59a370e74eaa869576855cc476f2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:38.807829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eef59a370e74eaa869576855cc476f2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:38.810844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:38.868757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eef59a370e74eaa869576855cc476f2', 'request_id': 'req-c5f9184b-3baf-4e11-ada1-512d2d5d2098'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.882974 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eef59a370e74eaa869576855cc476f2', 'request_id': 'req-c5f9184b-3baf-4e11-ada1-512d2d5d2098'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.890055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eef59a370e74eaa869576855cc476f2', 'request_id': 'req-c5f9184b-3baf-4e11-ada1-512d2d5d2098'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.905517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] The token cache is being invalidate because project 5eef59a370e74eaa869576855cc476f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:38.905817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.908447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c5f9184b-3baf-4e11-ada1-512d2d5d2098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:38.910619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1187/2337] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:38 2022] DELETE /identity/v3/projects/5eef59a370e74eaa869576855cc476f2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:38.915678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:38.915974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:38.916288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:38.964005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:38.965552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:39.001171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:39.138617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.140470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.140470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.151903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1151/2338] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:39.172573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:39.181834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:39.194300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1188/2339] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:39.198453 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.216284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2ba76-413b-46d0-9e04-e81b6205600d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.238701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.239897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.240186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.240477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.241253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:34:39.241569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.243549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.259877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1152/2340] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:34:39.264952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.282699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e4ba5-28e5-4602-9721-e8b2e13bb0fa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.290443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.291554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.292828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.293469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.294678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.297505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.356114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1189/2341] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:39 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:39.360518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.374681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e347a5-ad22-4994-b700-a4ba1ed83cad admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.524286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.525314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.525634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.525909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.526619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.529074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.554755 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1153/2342] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:39 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:34:39.556082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.570374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed6d6-922e-4c2a-a19c-dfa7107f45f3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.583213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.584568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.585955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.586414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.587316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.589507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.607737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1190/2343] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:39.614028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.626848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac215698-3d59-4c2d-8a17-46c5822c3294 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.634024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.634727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.635015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.635300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.636068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.638418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54770a3b-2d5e-40ea-9a8e-e96b7463ef98 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.649389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1154/2344] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:39.653626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.664219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44524d7e-93e8-441e-838d-c607cfcc7f9a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.669071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.670008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.670281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.670555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] PATH_INFO: `/v3/projects/1f178fe0a36f43babf15e0f2d5e79ce7/users/743245c4df3347acafb1d853c8129ea8/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.687616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f178fe0a36f43babf15e0f2d5e79ce7, user_id=743245c4df3347acafb1d853c8129ea8, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:39.690433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:39.713114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1191/2345] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:39 2022] PUT /identity/v3/projects/1f178fe0a36f43babf15e0f2d5e79ce7/users/743245c4df3347acafb1d853c8129ea8/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:34:39.723935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.724239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.724502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:39.779459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] MFA Rules not processed for user `743245c4df3347acafb1d853c8129ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:39.782477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:39.877105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1155/2346] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:39.889411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:39.891375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:39.946760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:39.984265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6071fb-937c-4a28-a845-82c9ba3536c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:39.988070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:39.991083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:39.991441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:39.991527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.002352 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:40.003937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9753af-fb95-431e-b206-d94c020208b7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:40.016924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1192/2347] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:40.060601 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:40.116338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:40.213573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60957a1b-8408-4c4e-80f0-10ce04ebd797 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:40.221821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:40.223000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.223370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.223665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] PATH_INFO: `/v3/users/743245c4df3347acafb1d853c8129ea8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.225143 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] RBAC: Authorizing `identity:delete_user(user_id=743245c4df3347acafb1d853c8129ea8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:40.228860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:40.308471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743245c4df3347acafb1d853c8129ea8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.320667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743245c4df3347acafb1d853c8129ea8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.321769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743245c4df3347acafb1d853c8129ea8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.342639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743245c4df3347acafb1d853c8129ea8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.352119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743245c4df3347acafb1d853c8129ea8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.361373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1156/2348] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:34:40 2022] DELETE /identity/v3/users/743245c4df3347acafb1d853c8129ea8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:40.365620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.366098 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.366181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.415245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:40.418072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:40.522363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1193/2349] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:40.529364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:40.530930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:40.534012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:40.535655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:40.549193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8396-aee0-4cc8-b9df-fd1736a8764d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:40.559397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:40.560756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.561055 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.561355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] PATH_INFO: `/v3/users/0cd5d671ef6b4672a5624534470d3aed` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.564901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd5d671ef6b4672a5624534470d3aed)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:40.572332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:40.626758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de6267-429b-45f2-9e37-03d1ab100a1d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:40.632139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:40.632874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.633219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.633596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.648906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:40.650855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14173806-7b4c-4bd8-8584-c60a56ed0bfb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:40.665085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1194/2350] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:40.668849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd5d671ef6b4672a5624534470d3aed'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.686175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd5d671ef6b4672a5624534470d3aed'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.687977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd5d671ef6b4672a5624534470d3aed'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.716468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd5d671ef6b4672a5624534470d3aed'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.724338 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd5d671ef6b4672a5624534470d3aed'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:40.732859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1157/2351] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:34:40 2022] DELETE /identity/v3/users/0cd5d671ef6b4672a5624534470d3aed => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:40.737485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.737852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.738212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.758802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:40.760118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:40.851327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1195/2352] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:40.866646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:40.868128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:40.880046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:40.957475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be87fb8-14d1-4d66-9710-2c03ef2b084f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:40.962866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:40.964074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:40.964517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:40.964954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:40.977017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:40.979286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:40.990128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1158/2353] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:41.089666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:41.092827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:41.139525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5770f8-7515-43e4-b38f-25a0f047cc73 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:41.149670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:41.153458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:41.153957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:41.154412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] PATH_INFO: `/v3/projects/1f178fe0a36f43babf15e0f2d5e79ce7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:41.158282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f178fe0a36f43babf15e0f2d5e79ce7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:41.164786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:41.180409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:41.261637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f178fe0a36f43babf15e0f2d5e79ce7', 'request_id': 'req-df7ac7ab-cd78-4533-8390-162cbe72ac07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.296205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f178fe0a36f43babf15e0f2d5e79ce7', 'request_id': 'req-df7ac7ab-cd78-4533-8390-162cbe72ac07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.310074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f178fe0a36f43babf15e0f2d5e79ce7', 'request_id': 'req-df7ac7ab-cd78-4533-8390-162cbe72ac07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.324784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] The token cache is being invalidate because project 1f178fe0a36f43babf15e0f2d5e79ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:41.325099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.326983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-df7ac7ab-cd78-4533-8390-162cbe72ac07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.329557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1196/2354] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:41 2022] DELETE /identity/v3/projects/1f178fe0a36f43babf15e0f2d5e79ce7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:41.646028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:41.648176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:41.662173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717a970-68c9-4c75-bcf8-6ab9761e5075 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:41.734289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:41.735163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:41.735444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:41.735997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] PATH_INFO: `/v3/projects/1b3a607b2e0346f2b7393c4a6ce31457` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:41.737784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b3a607b2e0346f2b7393c4a6ce31457)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:41.740235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:41.815050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b3a607b2e0346f2b7393c4a6ce31457', 'request_id': 'req-842c1d55-be08-4058-9789-fc2fda738e2c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.823715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b3a607b2e0346f2b7393c4a6ce31457', 'request_id': 'req-842c1d55-be08-4058-9789-fc2fda738e2c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.830932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b3a607b2e0346f2b7393c4a6ce31457', 'request_id': 'req-842c1d55-be08-4058-9789-fc2fda738e2c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.844784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] The token cache is being invalidate because project 1b3a607b2e0346f2b7393c4a6ce31457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:34:41.844979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.845812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-842c1d55-be08-4058-9789-fc2fda738e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:34:41.847591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1159/2355] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:34:41 2022] DELETE /identity/v3/projects/1b3a607b2e0346f2b7393c4a6ce31457 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:34:41.853248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:41.853815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:41.853903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:41.872719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:41.873957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:41.941322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1197/2356] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:41.950155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:41.950395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:41.950674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:41.971242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:41.972460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:41.980569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1160/2357] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:41.985597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:41.997185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9f045-017a-4c57-b0ac-dbf19df25626 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.001870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.002551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.002835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.003109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.003852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:34:42.004162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.005816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.023675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1198/2358] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:34:42.027834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:42.039898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50ab7-f90f-4351-80fe-85c3969cb723 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.044759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.045458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.045743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.046016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.046720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.048734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.084896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1161/2359] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:34:42 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:42.086590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:42.097401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098a08e-9806-4ef0-8e30-d8fa906e8375 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.164056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.164940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.165227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.165584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.166596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.170296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.196715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1199/2360] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:34:42 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:34:42.198609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:42.209696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920cc1c-c890-4df5-b5c3-e5154408034e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.214972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.215710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.216536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.216818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.218272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.219582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.234825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1162/2361] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:42.238172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:42.251991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867bc01-f616-43bb-9a9a-c5e3cfb63e5e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.257038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.257674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.257959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.258236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.269186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.270945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fae10-beb3-4b76-9b7d-2b4ee7e301c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.283842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1200/2362] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:34:42.287997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:42.294983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:42.303085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021edde-1ec6-4e53-8464-7e36d87f3902 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:42.309280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:42.310231 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.310513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.310788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] PATH_INFO: `/v3/projects/f2cacbef602540cbb16b13164dac4239/users/bb68bfb2472845f688d3e0076749fc29/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.326058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2cacbef602540cbb16b13164dac4239, user_id=bb68bfb2472845f688d3e0076749fc29, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:42.328715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba25d-5ee0-47f5-a22b-46e1ad06c993 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:42.342756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1163/2363] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:34:42 2022] PUT /identity/v3/projects/f2cacbef602540cbb16b13164dac4239/users/bb68bfb2472845f688d3e0076749fc29/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:34:42.353703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.354003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.354313 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.361806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:42.386501 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] MFA Rules not processed for user `bb68bfb2472845f688d3e0076749fc29`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:42.387994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:42.482405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1201/2364] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:34:42.906026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a7ce0-e228-4647-8b42-a333fe1bcc8c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:42.906606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a7ce0-e228-4647-8b42-a333fe1bcc8c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:42.906739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a7ce0-e228-4647-8b42-a333fe1bcc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:42.930899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e2a7ce0-e228-4647-8b42-a333fe1bcc8c None None] MFA Rules not processed for user `b22129ca8c714db6ad44dee94e6099e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:42.932109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e2a7ce0-e228-4647-8b42-a333fe1bcc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:42.991869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1164/2365] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:49.103343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:49.104793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:49.163049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70c8c6-e9c3-49c8-ae85-744cc8519a7c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:49.166663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:49.167315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:49.167598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:49.167877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:49.176452 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:49.177848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:49.187384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1202/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:34:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:34:52.985093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:34:57.241589 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e864e-c832-460e-b89d-bd3d0ad8d82d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:57.242159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e864e-c832-460e-b89d-bd3d0ad8d82d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:57.242159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e864e-c832-460e-b89d-bd3d0ad8d82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:57.259736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2a9e864e-c832-460e-b89d-bd3d0ad8d82d None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:34:57.261042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a9e864e-c832-460e-b89d-bd3d0ad8d82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:57.319781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1165/2367] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:34:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:34:57.377816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:34:57.379281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:34:57.455087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b4c0a-dcad-4cbd-9871-129f3ab6165f service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:34:57.460115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:34:57.460769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:34:57.461055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:34:57.461354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:34:57.470347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:34:57.471945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd539b8e-d14c-456b-8203-367760078d16 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:34:57.480201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1203/2368] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:35:07.312590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:07.476785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:09.341739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8d4e-6552-46a5-8c1d-5093ab9173d4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:09.342245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8d4e-6552-46a5-8c1d-5093ab9173d4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:09.342365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8d4e-6552-46a5-8c1d-5093ab9173d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:09.360219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-af1a8d4e-6552-46a5-8c1d-5093ab9173d4 None None] MFA Rules not processed for user `42165c1e1b324fedbb926ac20f0f920a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:09.361461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-af1a8d4e-6552-46a5-8c1d-5093ab9173d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:09.414734 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1166/2369] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:19.407943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:25.374367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b6dc9-a374-4373-abe0-3cf11d0f8220 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.375192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b6dc9-a374-4373-abe0-3cf11d0f8220 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.375976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b6dc9-a374-4373-abe0-3cf11d0f8220 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.377893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1204/2370] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:35:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:35:25.383648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295e2b-ddd7-476c-82a5-915732bb5c2d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.384178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295e2b-ddd7-476c-82a5-915732bb5c2d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.384296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295e2b-ddd7-476c-82a5-915732bb5c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.407216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-91295e2b-ddd7-476c-82a5-915732bb5c2d None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:25.408497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-91295e2b-ddd7-476c-82a5-915732bb5c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:25.495236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1167/2371] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:25.509360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b6dc9-a374-4373-abe0-3cf11d0f8220 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:25.526553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b6dc9-a374-4373-abe0-3cf11d0f8220 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:25.543323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:25.543323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.543323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.543323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.546508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:25.548733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3246-3d66-45c7-8f65-4dbe5218cb83 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:25.568413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1205/2372] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:25.660972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.661305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.661588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.662987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1168/2373] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:35:25.672520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75390-e967-4988-802d-041c6b9b1d5a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.673864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75390-e967-4988-802d-041c6b9b1d5a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.674297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c75390-e967-4988-802d-041c6b9b1d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.695112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22c75390-e967-4988-802d-041c6b9b1d5a None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:25.696360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22c75390-e967-4988-802d-041c6b9b1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:25.779441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1206/2374] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:25.789510 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:25.791473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:25.808267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07183f15-edfd-46ca-9834-9f183d2b9b4e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:25.813141 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:25.814212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.814566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.814871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.825301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:25.826655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aea318-9309-44de-9552-0175a961e3d4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:25.838377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1169/2375] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:25.969321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdd035-211a-40ae-906a-6bf974cc0262 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.969739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdd035-211a-40ae-906a-6bf974cc0262 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.970092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdd035-211a-40ae-906a-6bf974cc0262 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.971635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1207/2376] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:35:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:35:25.977204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:25.977576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:25.977854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:25.996514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:25.997762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:26.007584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1170/2377] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:26.022567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdd035-211a-40ae-906a-6bf974cc0262 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:26.034134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdd035-211a-40ae-906a-6bf974cc0262 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:26.037709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:26.038338 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:26.038618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:26.038892 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:26.049607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:26.051022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:26.062819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1208/2378] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:28.579552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:28.589981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d1cab-4730-4706-b1e1-e773979ff37f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:28.594452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:28.595105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:28.595384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:28.595664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:28.597644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:28.627020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:28.669885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:28.671389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33983131-2c58-4460-8ea2-0eea66f21ded service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:28.681616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1171/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:28.692983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9cd08-d3dd-460b-847a-4ea5f2cdc22d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:28.697742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:28.698636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:28.698940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:28.699740 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] PATH_INFO: `/v3/users/bb68bfb2472845f688d3e0076749fc29` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:28.701146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb68bfb2472845f688d3e0076749fc29)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:28.703107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:28.764236 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb68bfb2472845f688d3e0076749fc29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:28.772378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb68bfb2472845f688d3e0076749fc29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:28.787590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb68bfb2472845f688d3e0076749fc29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:28.788381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb68bfb2472845f688d3e0076749fc29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:28.796380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb68bfb2472845f688d3e0076749fc29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:28.802511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1209/2380] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:35:28 2022] DELETE /identity/v3/users/bb68bfb2472845f688d3e0076749fc29 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:35:28.807257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:28.807552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:28.807833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:28.828290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:28.829598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:28.892365 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1172/2381] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:28.902309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:28.903929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:28.909797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:28.968202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe13b9-bf9a-4a02-933c-ebaed266f98c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:28.972260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:28.972921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:28.973205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:28.973511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:28.983569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:28.985011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:28.995474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1210/2382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:29.391275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:29.393026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:29.433139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd5b6-4b55-4d18-88bd-e843dd497b0f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:29.449052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:29.450190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:29.450477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:29.450759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] PATH_INFO: `/v3/projects/f2cacbef602540cbb16b13164dac4239` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:29.452371 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2cacbef602540cbb16b13164dac4239)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:29.454733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:29.456612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:29.516416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2cacbef602540cbb16b13164dac4239', 'request_id': 'req-25f47c7f-ffe6-4383-bb49-533803ec9443'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:29.526076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2cacbef602540cbb16b13164dac4239', 'request_id': 'req-25f47c7f-ffe6-4383-bb49-533803ec9443'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:29.534301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2cacbef602540cbb16b13164dac4239', 'request_id': 'req-25f47c7f-ffe6-4383-bb49-533803ec9443'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:29.571017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] The token cache is being invalidate because project f2cacbef602540cbb16b13164dac4239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:35:29.571523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:29.572413 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-25f47c7f-ffe6-4383-bb49-533803ec9443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:35:29.585555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1173/2383] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:35:29 2022] DELETE /identity/v3/projects/f2cacbef602540cbb16b13164dac4239 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:35:33.546736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:33.555669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ea4b9-5301-4954-8626-606121617b67 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:33.609268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:33.609968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:33.610251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:33.610525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:33.614126 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:33.686221 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:33.687700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:33.697722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1211/2384] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:35:34.676200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17865b-3fbe-4992-b0b2-addfea159501 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.676700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17865b-3fbe-4992-b0b2-addfea159501 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.676909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17865b-3fbe-4992-b0b2-addfea159501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.696084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8e17865b-3fbe-4992-b0b2-addfea159501 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:35:34.697508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e17865b-3fbe-4992-b0b2-addfea159501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:34.754973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1174/2385] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:35:34.768375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:35:34.769832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:35:34.834054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:34.851034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3545b5-c4ef-4a54-b34c-abec3b4ab2a6 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:35:34.857453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:35:34.858038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:35:34.858322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:35:34.858593 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:35:34.868752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:35:34.870392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc422f9-37e7-491e-9051-32853423f304 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:35:34.878715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1212/2386] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:35:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:35:44.747957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:35:44.876021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:36:04.690935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:36:04.691419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:36:04.691621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:36:04.711013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:36:04.712222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:36:04.717232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad09e1-f95d-4d11-9746-7ce518fd309c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:36:04.717739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad09e1-f95d-4d11-9746-7ce518fd309c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:36:04.717857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad09e1-f95d-4d11-9746-7ce518fd309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:36:04.720598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1175/2387] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:36:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:36:04.736008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6fad09e1-f95d-4d11-9746-7ce518fd309c None None] MFA Rules not processed for user `a48066ed22e14c64a316fe4d66394f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:36:04.737086 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6fad09e1-f95d-4d11-9746-7ce518fd309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:36:04.743845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:36:04.756903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889dc49-de5d-4ec8-a1d0-0d89a3f82f60 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:36:04.763016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:36:04.763796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:36:04.764151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:36:04.764505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:36:04.773667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:36:04.775247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:36:04.784693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1176/2388] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:36:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:36:04.798389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1213/2389] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:36:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:36:06.006299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:36:06.007722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:36:06.087019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aeb71-4123-43aa-88ea-fa74911137da service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:36:06.092639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:36:06.093325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:36:06.093693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:36:06.093981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:36:06.095793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:36:06.108470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:36:06.110105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a01be6-ab3b-452b-bd17-f0dae8da5edb service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:36:06.120481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1177/2390] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:36:14.790214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:36:16.116649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=41 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:36:21.319939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:36:21.320475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:36:21.320475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:36:21.339520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] MFA Rules not processed for user `b22129ca8c714db6ad44dee94e6099e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:36:21.340609 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:36:21.394266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1214/2391] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:36:31.387529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:04.397642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:04.398167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:04.398281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:04.416753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] MFA Rules not processed for user `3753498f22d5443ab0c01473c91a263c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:04.418426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:04.472619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1178/2392] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:37:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:14.466053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:26.371484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:26.372823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:26.384438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f54b4e-28ea-4c8e-bdfc-bfe620fb2feb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:26.387936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:26.388569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:26.388848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:26.389122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:26.390843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:26.466061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:26.467485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:26.478712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1215/2393] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:29.135829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:29.137224 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:29.153503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda338-bb44-4d21-86f9-468cc651b529 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:29.159204 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:29.159837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:29.160117 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:29.160391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:29.162286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:29.173017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:29.175042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32278c4-15ac-4ea3-8d72-b627f64b754e service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:29.184384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1179/2394] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:31.775957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:31.777421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:31.788460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd3cd3-8d10-41b0-b527-3c2db0bc1676 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:31.791991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:31.792628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:31.792909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:31.793184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:31.795137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:31.811007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:31.812684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337765e-e363-47b5-8603-78e4d60843c9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:31.824756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1216/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:37:36.040978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:36.041529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:36.041627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:36.059617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] MFA Rules not processed for user `b22129ca8c714db6ad44dee94e6099e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:36.060849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:36.068247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1180/2396] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:36.083546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:36.083972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:36.084131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:36.101171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] MFA Rules not processed for user `a48066ed22e14c64a316fe4d66394f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:37:36.102992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:36.110444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1217/2397] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:37:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:37:46.054286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=9 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:46.096209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=43 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:48.562056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:48.572312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a84a-2c9b-448e-bd8c-954ebf15493f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:48.575835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:48.576468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:48.576748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:48.577019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:48.578774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:48.588223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:48.589669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:48.597663 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1181/2398] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:53.138426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:53.151489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca911d94-0138-4ce9-89ac-72c95d31f6a8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:53.161877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:53.162563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:53.163067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:53.163353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:53.165835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:53.178423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:53.180033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141fe2f-19f7-4099-8aaf-3aba4e1e1b2c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:53.191147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1218/2399] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:37:58.594114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:37:58.627325 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:37:58.629059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:58.645900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bb755-985c-4716-9c7c-3a3c786a0b6b service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:37:58.652089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:37:58.653268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:37:58.654091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:37:58.654618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:37:58.656657 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:37:58.670857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:37:58.672882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:37:58.696230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1182/2400] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:37:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:00.629264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:00.629841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:00.629957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:00.647267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] MFA Rules not processed for user `42165c1e1b324fedbb926ac20f0f920a`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:00.648444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:00.702287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1219/2401] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:38:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:01.015258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:01.024912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776222e-a76c-4b7d-bf5b-95e283a1783e service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:01.028490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:01.029127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:01.029452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:01.029750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:01.031495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:01.040778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:01.042222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:01.050138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1183/2402] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:01.481011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:01.492900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d144d-a723-4b02-a583-f05493c39e99 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:01.498449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:01.499115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:01.499522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:01.499810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:01.501673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:01.513911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:01.515535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:01.531260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1220/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:01.714573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:01.727588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a302d1-849d-43f8-89c4-99acf4d5ac41 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:01.733120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:01.733748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:01.734053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:01.734349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:01.736076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:01.746709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:01.748283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7363c-bb0f-4efe-85a3-e712d1bd96b5 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:01.757769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1184/2404] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:11.526826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=37 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:38:11.753519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=18 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:38:24.731009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:24.744462 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85afaf3-b3e3-469e-9188-fb4e8736eea6 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:24.750193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:24.750848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:24.751248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:24.751507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:24.753314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:24.766010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:24.767599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:24.779251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1221/2405] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:29.331400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271ab6-3b59-4332-bc96-179350f43ff4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:29.331903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271ab6-3b59-4332-bc96-179350f43ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:29.332001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f271ab6-3b59-4332-bc96-179350f43ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:29.349532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f271ab6-3b59-4332-bc96-179350f43ff4 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:29.350805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f271ab6-3b59-4332-bc96-179350f43ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:29.359879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1185/2406] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:29.406029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:29.418882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1099e8-4484-47b9-9874-345fdfc50d3b service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:29.423997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:29.424645 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:29.424931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:29.425214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:29.440350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:29.441978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c70939-21da-4aaf-ba9f-7ac76040469a service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:29.450489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1222/2407] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:38:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:38:39.344595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:38:39.447397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=12 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:38:43.787499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:43.788053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:43.788685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:43.790029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1186/2408] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:38:43.796261 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc686d-9b6f-487e-9cd0-444811838071 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:43.796714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc686d-9b6f-487e-9cd0-444811838071 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:43.796832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc686d-9b6f-487e-9cd0-444811838071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:43.816056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1bbc686d-9b6f-487e-9cd0-444811838071 None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:43.817395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1bbc686d-9b6f-487e-9cd0-444811838071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:43.827080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1223/2409] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:43.838450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:43.840133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:43.902301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea00db-e13a-429f-af5a-23da8fa0c933 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:43.906087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:43.906748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:43.907030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:43.907299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:43.918403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:43.919902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10ae2c-9f57-45b4-9971-022c38f277df service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:43.931358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1187/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:43.996225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9170f0-8347-479d-9af7-c1deb9265171 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:43.996543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9170f0-8347-479d-9af7-c1deb9265171 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:43.996822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9170f0-8347-479d-9af7-c1deb9265171 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:43.998237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1224/2411] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:38:44.003010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921037b7-e5b4-42f0-a1de-31233e9de0a8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:44.003301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921037b7-e5b4-42f0-a1de-31233e9de0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:44.003582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921037b7-e5b4-42f0-a1de-31233e9de0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:44.022003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-921037b7-e5b4-42f0-a1de-31233e9de0a8 None None] MFA Rules not processed for user `2b899731a1ef4d9b961492dd78632ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:44.023227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-921037b7-e5b4-42f0-a1de-31233e9de0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:44.029684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1188/2412] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:38:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:44.041171 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9170f0-8347-479d-9af7-c1deb9265171 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:44.051940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9170f0-8347-479d-9af7-c1deb9265171 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:44.056048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:44.056679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:44.056964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:44.057244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:44.064438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:44.065920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd56932-fa64-4133-a1f3-cd92f59bfb82 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:44.075699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1225/2413] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:38:44.159281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07603d42-5124-418f-aafd-d165cf005ee5 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:44.159572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07603d42-5124-418f-aafd-d165cf005ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:44.159854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07603d42-5124-418f-aafd-d165cf005ee5 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:44.161219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1189/2414] 10.222.0.25 () {54 vars in 921 bytes} [Tue Jul 12 14:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 14:38:44.166040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6836d6-4e9b-4cb8-b846-641379e7b3ce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:44.166382 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6836d6-4e9b-4cb8-b846-641379e7b3ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:44.166666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6836d6-4e9b-4cb8-b846-641379e7b3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:44.184880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-da6836d6-4e9b-4cb8-b846-641379e7b3ce None None] MFA Rules not processed for user `a1fb1196840240b4aa05a2f83d4c7ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:44.186107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-da6836d6-4e9b-4cb8-b846-641379e7b3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:44.196122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1226/2415] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Jul 12 14:38:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:44.206373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07603d42-5124-418f-aafd-d165cf005ee5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:44.217312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07603d42-5124-418f-aafd-d165cf005ee5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:44.220818 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:44.221470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:44.221747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:44.222021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:44.232186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:44.233636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:44.245821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1190/2416] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:38:49.122447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:49.123020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:49.123020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:49.142264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:38:49.143475 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:38:49.150745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1227/2417] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:38:49.162577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:38:49.174153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ce112-6b1c-4ceb-bfa2-d00eb30055ed service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:38:49.180007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:38:49.180632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:38:49.180916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:38:49.181191 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:38:49.189442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:38:49.190996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:38:49.199092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1191/2418] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:38:59.135709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=15 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:38:59.195863 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=41 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:04.457818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:04.468559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82908-875b-4b0d-964e-440658fb14e1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:04.472136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:04.472905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:04.473234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:04.473617 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:04.476544 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:04.490199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:04.491949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:04.501820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1228/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:07.021460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:07.031085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51821e9f-c83e-492c-8849-d9c890a9fdec service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:07.034795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:07.035438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:07.035731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:07.036015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:07.039180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:07.106074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:07.107576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:07.118015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1192/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:10.494891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:10.508229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79997b2-12f8-41af-be79-83ae41b72d87 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:10.514103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:10.514956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:10.515390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:10.515665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:10.517682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:10.530993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:10.532687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f485988-75bd-4c26-8eaa-f419ab6e78df service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:10.543443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1229/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:10.565222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:10.567016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:10.585760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1244c-e29a-4aeb-8bf3-63e0e6476b34 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:10.592671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:10.593702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:10.594057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:10.594419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] PATH_INFO: `/v3/users/a48066ed22e14c64a316fe4d66394f44` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:10.595703 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48066ed22e14c64a316fe4d66394f44)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:10.597981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:10.646036 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48066ed22e14c64a316fe4d66394f44'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:10.653390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48066ed22e14c64a316fe4d66394f44'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:10.653999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48066ed22e14c64a316fe4d66394f44'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:10.666815 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48066ed22e14c64a316fe4d66394f44'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:10.673671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48066ed22e14c64a316fe4d66394f44'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:10.680307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1193/2422] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:10 2022] DELETE /identity/v3/users/a48066ed22e14c64a316fe4d66394f44 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:10.746203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:10.746498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:10.746884 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:10.765291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:10.766538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:10.824463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1230/2423] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:10.838256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:10.839957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:10.863815 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:10.865430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:10.923569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778ec44-4eda-4abf-901c-c8ee301fc4e4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:10.928826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:10.929513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:10.929803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:10.930085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:10.936221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:10.939165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:10.940944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1bd8-4f96-4bf1-8b43-99ec6e0b700f service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:10.942566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdf208-3712-423f-8172-05cde49855d3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:10.947238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:10.948111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:10.948380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:10.948655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] PATH_INFO: `/v3/projects/926f1338d73b48aebc0431e2121a0dba` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:10.949960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926f1338d73b48aebc0431e2121a0dba)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:10.951864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:10.952269 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1194/2424] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:39:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:39:11.005258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926f1338d73b48aebc0431e2121a0dba', 'request_id': 'req-1a6d63dd-d795-46f5-9deb-1571c5d05b01'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:11.013536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926f1338d73b48aebc0431e2121a0dba', 'request_id': 'req-1a6d63dd-d795-46f5-9deb-1571c5d05b01'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:11.023426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926f1338d73b48aebc0431e2121a0dba', 'request_id': 'req-1a6d63dd-d795-46f5-9deb-1571c5d05b01'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:11.034887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] The token cache is being invalidate because project 926f1338d73b48aebc0431e2121a0dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:11.035181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:11.035851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1a6d63dd-d795-46f5-9deb-1571c5d05b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:11.037710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1231/2425] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:10 2022] DELETE /identity/v3/projects/926f1338d73b48aebc0431e2121a0dba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:11.042929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.043341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.043697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.063898 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:11.065439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:11.134830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1195/2426] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:11.143426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.143723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.144007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.164784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:11.166120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:11.174797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1232/2427] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:11.179624 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.190825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c286d-724e-4272-a9e1-a60f372e779e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.195943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.197088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.197554 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.198092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.198997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:11.199719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.201966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.215938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1196/2428] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:11.220013 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.225483 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:11.230961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7f9ea-b180-4740-a723-0fd079ff698f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.238064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.238961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.239493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.239956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.242805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.244794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.272439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1233/2429] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:11.276498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.286747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5068463-9fbf-4c77-b656-047f7321b661 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.357981 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.358764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.359123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.359462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.360252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.362300 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.387775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1197/2430] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:11 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:11.391709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.403225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd7a2c-e5a6-48e0-b3ec-73b5ac394380 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.408434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.409106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.409416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.409720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.410447 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.412057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.423522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1234/2431] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:11.427616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.439255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d417b-142c-4b3b-a8a0-42576b06798d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.444495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.445163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.445566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.445851 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.446611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.448214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bf75b-c23f-44d3-99ac-fa36638b05c9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.459892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1198/2432] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:11.463864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.474847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8d5db-c466-4029-b3dc-f5b86fd05922 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.483591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.484714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.485217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.485666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] PATH_INFO: `/v3/projects/decc10c5e64842e38d528c832429351c/users/8329adf3b3b14eb488f0473e2b61e48b/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.502981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] RBAC: Authorizing `identity:create_grant(project_id=decc10c5e64842e38d528c832429351c, user_id=8329adf3b3b14eb488f0473e2b61e48b, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.506368 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.536167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1235/2433] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:11 2022] PUT /identity/v3/projects/decc10c5e64842e38d528c832429351c/users/8329adf3b3b14eb488f0473e2b61e48b/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:11.547278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.547720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.548311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.576805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] MFA Rules not processed for user `8329adf3b3b14eb488f0473e2b61e48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:11.578051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:11.626933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:11.645961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1199/2434] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:11.657208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:11.659040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:11.737733 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc8bc-c91e-41ec-bc3b-93f5e900365c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:11.743144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:11.743788 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:11.744072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:11.744348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:11.752548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:11.754162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eef9d-bad0-4d98-8b4e-dea654ff7d55 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:11.763143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1236/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:19.041840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:19.104030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42f8a3-87e0-4798-88af-cc07f693e512 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:19.108897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:19.109807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.110096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.110380 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] PATH_INFO: `/v3/users/8329adf3b3b14eb488f0473e2b61e48b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.111575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8329adf3b3b14eb488f0473e2b61e48b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:19.113665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:19.159137 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8329adf3b3b14eb488f0473e2b61e48b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.168313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8329adf3b3b14eb488f0473e2b61e48b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.168977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8329adf3b3b14eb488f0473e2b61e48b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.181262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8329adf3b3b14eb488f0473e2b61e48b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.188386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8329adf3b3b14eb488f0473e2b61e48b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.194548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1200/2436] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:19 2022] DELETE /identity/v3/users/8329adf3b3b14eb488f0473e2b61e48b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:19.198927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.199416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.199509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.216590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:19.217860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.277706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1237/2437] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:19.287280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:19.288832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.347612 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922663fc-b51c-4b5b-a4f8-82f49fff5332 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:19.351238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:19.351873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.352152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.352538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.361447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:19.368198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c91b64-42f2-4a0d-80ca-cf8488126ded service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:19.378101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1201/2438] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:19.831347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:19.832766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.845110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bc0b-198d-45f8-92f6-e6060f85de21 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:19.852319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:19.853194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.853537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.853824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] PATH_INFO: `/v3/projects/decc10c5e64842e38d528c832429351c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.855044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decc10c5e64842e38d528c832429351c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:19.856973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:19.900268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:19.924347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decc10c5e64842e38d528c832429351c', 'request_id': 'req-1181deca-0378-4073-b652-4af0da20cf38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.933467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decc10c5e64842e38d528c832429351c', 'request_id': 'req-1181deca-0378-4073-b652-4af0da20cf38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.940099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decc10c5e64842e38d528c832429351c', 'request_id': 'req-1181deca-0378-4073-b652-4af0da20cf38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.951768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] The token cache is being invalidate because project decc10c5e64842e38d528c832429351c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:19.952139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.952858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1181deca-0378-4073-b652-4af0da20cf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:19.954495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1238/2439] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:19 2022] DELETE /identity/v3/projects/decc10c5e64842e38d528c832429351c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:19.959903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:19.960217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:19.960507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:19.979105 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:19.980294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:19.989072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:20.049109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1202/2440] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:20.058033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.058326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.058611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.087040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:20.088272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:20.096284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1239/2441] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:20.101061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.110960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119077e2-ea71-4c3b-9386-d0431b5fbf33 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.115290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.115943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.116228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.116507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.117259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:20.117605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.119240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.132188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1203/2442] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:20.136424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.147410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027db2b1-2167-4100-9d57-70660678ee04 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.152181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.152847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.153131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.153439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.154139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.156248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.185993 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1240/2443] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:20.190432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.201030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68f46-54e5-42bf-a33b-f218dd7bb6d1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.258647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.259322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.259608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.259888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.260660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.262604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.292552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1204/2444] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:20.293902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.305776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2861a-323b-4a3e-911b-e6d8eda55c8b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.311758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.312669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.313422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.313778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.314598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.316346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.332233 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1241/2445] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:20.333497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.343987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489bd52-c332-4eed-aad2-73941173ad02 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.348817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.349520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.350326 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.350619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.351352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.352986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35948-9032-45d0-ba1c-6429792c4f7f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.363268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1205/2446] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:20.367649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.377991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad2cbc-56a5-4eba-8084-12d8c8fc9bd7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.383092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.383970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.384249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.384528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] PATH_INFO: `/v3/projects/64469e9883d34f7e91ab99ccde3f017a/users/f5c4c4a602b54153b0810b0880c8d9e1/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.400446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64469e9883d34f7e91ab99ccde3f017a, user_id=f5c4c4a602b54153b0810b0880c8d9e1, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.404033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.420819 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1242/2447] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:20 2022] PUT /identity/v3/projects/64469e9883d34f7e91ab99ccde3f017a/users/f5c4c4a602b54153b0810b0880c8d9e1/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:20.431083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.431452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.431738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.461501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] MFA Rules not processed for user `f5c4c4a602b54153b0810b0880c8d9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:20.462828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:20.533058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1206/2448] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:20.537685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.633998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84850153-043f-459d-aabb-8d4812e81159 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.639905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.640570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.640880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.641159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.641910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.643802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.678631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1243/2449] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:20.682599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.699908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ef66b-2c34-4cd1-874f-b453f20bd9b6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.705756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:20.788010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.788692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.788972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.789301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.790040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.792098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.818477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1207/2450] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:20 2022] POST /identity/v3/users => generated 309 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:20.822852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.835925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a55ed-ee05-445b-b666-5c7aa59a2672 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.841414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.842293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.842715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.843060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.843988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.845773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.857749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1244/2451] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:20.861924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.872629 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd10d5-ef80-4448-8718-d845f348fda3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.877544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.878530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.878876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.879219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] PATH_INFO: `/v3/projects/92dc68e21c154fd88cdf063f82441fe8/users/0298fa7e98004b7aa8be3c165e699052/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.896165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dc68e21c154fd88cdf063f82441fe8, user_id=0298fa7e98004b7aa8be3c165e699052, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.899054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:20.914587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1208/2452] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:20 2022] PUT /identity/v3/projects/92dc68e21c154fd88cdf063f82441fe8/users/0298fa7e98004b7aa8be3c165e699052/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:20.918604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:20.929285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a4354-a94e-4a81-bff2-0ed67d46370c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:20.949844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:20.989383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:20.990058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:20.990337 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:20.991106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:20.991825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:20.993438 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55b6e8-c126-40b8-baac-42dc8f99002e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.007392 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1245/2453] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:21.008532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.020258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e271c87-4685-4c70-909b-d5d44d0f4417 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.024879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.025876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.026162 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.026440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] PATH_INFO: `/v3/projects/92dc68e21c154fd88cdf063f82441fe8/users/0298fa7e98004b7aa8be3c165e699052/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.028458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dc68e21c154fd88cdf063f82441fe8, user_id=0298fa7e98004b7aa8be3c165e699052, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.031179 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.042218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1209/2454] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:21 2022] PUT /identity/v3/projects/92dc68e21c154fd88cdf063f82441fe8/users/0298fa7e98004b7aa8be3c165e699052/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:21.052824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.053639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.053927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.082644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] MFA Rules not processed for user `0298fa7e98004b7aa8be3c165e699052`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:21.083873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:21.168767 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1246/2455] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:21.180828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.183509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:21.264508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa80a51-ecfb-41f3-8aa8-15c02190c694 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.269751 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.270408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.270695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.270973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.280991 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.282463 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76713c3-138f-4773-94a8-06ae6b5dd447 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.293037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1210/2456] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:21.447314 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.512777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7acad-3de5-4775-8168-4829470959e8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.518065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.520800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.520800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.520800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] PATH_INFO: `/v3/users/f5c4c4a602b54153b0810b0880c8d9e1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.521118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c4c4a602b54153b0810b0880c8d9e1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.523359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.576839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c4c4a602b54153b0810b0880c8d9e1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:21.579834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:21.593614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c4c4a602b54153b0810b0880c8d9e1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:21.606624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c4c4a602b54153b0810b0880c8d9e1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:21.607291 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c4c4a602b54153b0810b0880c8d9e1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:21.614349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c4c4a602b54153b0810b0880c8d9e1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:21.622199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1247/2457] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:21 2022] DELETE /identity/v3/users/f5c4c4a602b54153b0810b0880c8d9e1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:21.626643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.626935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.627214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.644760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:21.645989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:21.712270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1211/2458] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:21.722621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:21.724144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:21.782081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4459816-8e4e-4089-8c1a-123d5e2eb2c4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:21.785708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:21.786355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:21.786639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:21.786917 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:21.796460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:21.797952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:21.808128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1248/2459] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:22.084842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.086314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.094495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:22.099138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badfee6-01fc-4d54-ae80-0a40b48ce614 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.105304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.106258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.106609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.106957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] PATH_INFO: `/v3/projects/64469e9883d34f7e91ab99ccde3f017a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.108252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64469e9883d34f7e91ab99ccde3f017a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.110549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.162214 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64469e9883d34f7e91ab99ccde3f017a', 'request_id': 'req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.170116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64469e9883d34f7e91ab99ccde3f017a', 'request_id': 'req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.175944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64469e9883d34f7e91ab99ccde3f017a', 'request_id': 'req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.187520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] The token cache is being invalidate because project 64469e9883d34f7e91ab99ccde3f017a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:22.188366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.188678 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.190245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1212/2460] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:22 2022] DELETE /identity/v3/projects/64469e9883d34f7e91ab99ccde3f017a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:22.194456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.204504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259d691-ce65-4c87-b8f5-e5f4da43b1a3 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.268417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.269576 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.269709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.269984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] PATH_INFO: `/v3/users/0298fa7e98004b7aa8be3c165e699052` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.271177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0298fa7e98004b7aa8be3c165e699052)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.273652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.321006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0298fa7e98004b7aa8be3c165e699052'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.331960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0298fa7e98004b7aa8be3c165e699052'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.344425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:22.345990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0298fa7e98004b7aa8be3c165e699052'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.346699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0298fa7e98004b7aa8be3c165e699052'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.355956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b64614db-40ee-4fae-ae18-c5b9ea4ec024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0298fa7e98004b7aa8be3c165e699052'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.362116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1249/2461] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:22 2022] DELETE /identity/v3/users/0298fa7e98004b7aa8be3c165e699052 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:22.687151 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:22.689488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.710016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284b78-bffa-4d5c-a4dc-c02a951f8c8d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:22.772482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:22.773358 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.777646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.778004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] PATH_INFO: `/v3/projects/92dc68e21c154fd88cdf063f82441fe8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.779315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92dc68e21c154fd88cdf063f82441fe8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:22.781246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:22.829590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92dc68e21c154fd88cdf063f82441fe8', 'request_id': 'req-e8400b96-a026-4bae-bebd-6720faa296c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.838696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92dc68e21c154fd88cdf063f82441fe8', 'request_id': 'req-e8400b96-a026-4bae-bebd-6720faa296c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.846051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92dc68e21c154fd88cdf063f82441fe8', 'request_id': 'req-e8400b96-a026-4bae-bebd-6720faa296c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.859064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] The token cache is being invalidate because project 92dc68e21c154fd88cdf063f82441fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:22.859343 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.860006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e8400b96-a026-4bae-bebd-6720faa296c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:22.861591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1213/2462] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:22 2022] DELETE /identity/v3/projects/92dc68e21c154fd88cdf063f82441fe8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:22.866574 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.867050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.867146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.888052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:22.889292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.954267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1250/2463] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:22.962816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:22.963313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:22.963693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:22.984193 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:22.986183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:22.997406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1214/2464] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:23.002048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.012342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628f0cf-46f1-47d7-bea4-317c97f18e89 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.017607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.018456 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.018786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.019127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.020092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:23.020329 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.023028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.035581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1251/2465] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:23.040919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.050939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7589c0-7caf-4888-87de-ccb2410723e5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.056146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.057317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.057549 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.057878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.058766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.061071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.072490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:23.092454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1215/2466] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:23.096619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.107327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcc8c-57ac-44b5-888a-382b43254669 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.165172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.165783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.166065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.166339 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.167028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.168919 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.189679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1252/2467] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:23.193866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.204900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1d646-95ee-4c8e-9bca-87bcad3d5790 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.209555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.210211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.210495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.210772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.211486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.213064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.222927 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1216/2468] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.226957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.236978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5a9c3-8bf7-4eb2-ac94-d99f864cca33 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.242087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.242734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.243014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.243287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.244003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.245635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fc2fd-6bcc-4d0a-b8d5-b75904d31ff3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.257543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1253/2469] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.261756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.271813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e20c19-c38a-400e-a85c-81b606cad5d9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.276596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.277566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.277849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.278126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] PATH_INFO: `/v3/projects/d8b575771fc84e478bafe4ee2998a415/users/c6ba1f426a3c4fceb2b08cee12439576/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.293235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b575771fc84e478bafe4ee2998a415, user_id=c6ba1f426a3c4fceb2b08cee12439576, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.295939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.309837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1217/2470] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:23 2022] PUT /identity/v3/projects/d8b575771fc84e478bafe4ee2998a415/users/c6ba1f426a3c4fceb2b08cee12439576/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:23.321833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.322130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.322409 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.349575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] MFA Rules not processed for user `c6ba1f426a3c4fceb2b08cee12439576`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:23.351449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:23.395320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:23.419920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1254/2471] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:23.424528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.488469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489401ed-b35c-42bc-b6cf-a42b09b3b30a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.493783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.494442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.494728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.495013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.495722 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.497686 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.525156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1218/2472] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:23.529297 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.540197 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d797ca-75be-4042-97e3-ad96212c6c81 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.597102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.597796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.598081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.598361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.599053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.600945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.620110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1255/2473] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/users => generated 306 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:23.624092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.635350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a5e43-578f-48e9-a379-afea17d02c17 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.640021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.640680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.640967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.641247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.641999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.643595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.654627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1219/2474] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.658669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.668787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798a68d-0436-4c3a-b846-8615bc5e5972 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.673573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.674459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.674745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.675024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] PATH_INFO: `/v3/projects/dbab2d63c0f74c078e75fd084fd496ea/users/511e8d4c0a964f75a7b29d86e3df318b/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.690758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbab2d63c0f74c078e75fd084fd496ea, user_id=511e8d4c0a964f75a7b29d86e3df318b, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.693525 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.706957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1256/2475] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:23 2022] PUT /identity/v3/projects/dbab2d63c0f74c078e75fd084fd496ea/users/511e8d4c0a964f75a7b29d86e3df318b/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:23.711039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.721508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279362b3-251f-4445-8b23-74527fbb47e8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.780543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.781210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.781546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.781825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.782588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.784175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dc3e6-58e0-4454-baa7-13cc52e8630a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.793902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1220/2476] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:23.798076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.809048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132530e-b1b5-4f7d-a97e-1c0b0dee0247 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:23.813937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:23.814820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.815107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.815386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] PATH_INFO: `/v3/projects/dbab2d63c0f74c078e75fd084fd496ea/users/511e8d4c0a964f75a7b29d86e3df318b/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.817361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbab2d63c0f74c078e75fd084fd496ea, user_id=511e8d4c0a964f75a7b29d86e3df318b, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:23.820559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:23.830846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1257/2477] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:23 2022] PUT /identity/v3/projects/dbab2d63c0f74c078e75fd084fd496ea/users/511e8d4c0a964f75a7b29d86e3df318b/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:23.842022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:23.842400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:23.842747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:23.871971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] MFA Rules not processed for user `511e8d4c0a964f75a7b29d86e3df318b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:23.873196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:23.885449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:23.942990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1221/2478] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:23.954779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:23.956549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:24.012858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a9d90-cdd1-43c5-b0ab-d9f8f6acff3e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:24.016692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:24.017267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:24.017597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:24.017879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:24.027065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:24.028509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:24.038182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1258/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:24.114739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:24.119103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:24.137288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fcaa4-e8bd-4cda-907b-1fae804f519b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:24.140980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:24.141655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:24.141940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:24.142218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:24.210740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:24.212252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:24.225505 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1222/2480] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:26.648601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:26.650024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:26.691938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:26.708321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f08732-ae0a-4405-ae4b-7392aa3b5236 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:26.712027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:26.712670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:26.712958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:26.713238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:26.715046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:26.776379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:26.777861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0aebba-485f-4c83-9b49-7ad6c911dcb8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:26.787575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1259/2481] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:28.278336 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.288983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b5a5e-5cf8-48dd-8542-aa9074fff751 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.293504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.294438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.295146 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.295428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] PATH_INFO: `/v3/users/c6ba1f426a3c4fceb2b08cee12439576` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.296615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ba1f426a3c4fceb2b08cee12439576)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.298576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.344760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ba1f426a3c4fceb2b08cee12439576'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.352745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ba1f426a3c4fceb2b08cee12439576'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.353685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ba1f426a3c4fceb2b08cee12439576'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.366774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ba1f426a3c4fceb2b08cee12439576'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.373671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ba1f426a3c4fceb2b08cee12439576'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.379806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1223/2482] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:28 2022] DELETE /identity/v3/users/c6ba1f426a3c4fceb2b08cee12439576 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:28.384381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.384846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.384942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.402242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:28.403458 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.466868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1260/2483] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:28.477793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.479316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.519470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:28.560345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f1d14-c625-4669-9715-67b00c910e12 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.564379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.565030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.565310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.565615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.577032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.578483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.591824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1224/2484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:28.895104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:28.896731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:28.908967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1649e-a093-4e33-aae8-351a6028a257 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:28.914334 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:28.915408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:28.915695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:28.915977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] PATH_INFO: `/v3/projects/d8b575771fc84e478bafe4ee2998a415` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:28.917262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b575771fc84e478bafe4ee2998a415)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:28.919699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:28.984473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b575771fc84e478bafe4ee2998a415', 'request_id': 'req-f4440543-65a4-49f1-a29a-9f55d8586c69'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:28.994265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b575771fc84e478bafe4ee2998a415', 'request_id': 'req-f4440543-65a4-49f1-a29a-9f55d8586c69'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.003032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b575771fc84e478bafe4ee2998a415', 'request_id': 'req-f4440543-65a4-49f1-a29a-9f55d8586c69'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.019602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] The token cache is being invalidate because project d8b575771fc84e478bafe4ee2998a415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:29.019895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.021992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.027022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1261/2485] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:28 2022] DELETE /identity/v3/projects/d8b575771fc84e478bafe4ee2998a415 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:29.028196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.040089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfada6-9794-4c11-b6ca-6e249c836099 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:29.110604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.111475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.111760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.112035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] PATH_INFO: `/v3/users/511e8d4c0a964f75a7b29d86e3df318b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.113283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=511e8d4c0a964f75a7b29d86e3df318b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.115239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.162835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511e8d4c0a964f75a7b29d86e3df318b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.170903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511e8d4c0a964f75a7b29d86e3df318b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.171580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511e8d4c0a964f75a7b29d86e3df318b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.184349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511e8d4c0a964f75a7b29d86e3df318b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.191452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2985559a-379a-4672-b085-6c781ef0ef14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511e8d4c0a964f75a7b29d86e3df318b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.197555 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1225/2486] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:29 2022] DELETE /identity/v3/users/511e8d4c0a964f75a7b29d86e3df318b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:29.498265 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.499649 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.517415 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4440543-65a4-49f1-a29a-9f55d8586c69 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:29.530371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:29.596477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.597027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.597309 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.597614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] PATH_INFO: `/v3/projects/dbab2d63c0f74c078e75fd084fd496ea` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.598814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbab2d63c0f74c078e75fd084fd496ea)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.600748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.656118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbab2d63c0f74c078e75fd084fd496ea', 'request_id': 'req-20359b4d-db90-436f-bbb5-97f013a9245b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.665594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbab2d63c0f74c078e75fd084fd496ea', 'request_id': 'req-20359b4d-db90-436f-bbb5-97f013a9245b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.673137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbab2d63c0f74c078e75fd084fd496ea', 'request_id': 'req-20359b4d-db90-436f-bbb5-97f013a9245b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.688227 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] The token cache is being invalidate because project dbab2d63c0f74c078e75fd084fd496ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:29.688520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.689205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20359b4d-db90-436f-bbb5-97f013a9245b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:29.690823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1262/2487] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:29 2022] DELETE /identity/v3/projects/dbab2d63c0f74c078e75fd084fd496ea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:29.696417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.696841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.696934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.716716 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:29.717979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.758677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:29.781941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1226/2488] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:29.790786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.791080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.793688 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.814203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:29.828363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:29.839418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1263/2489] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:29.843312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.854395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d5284-56ea-415e-8196-998660616f79 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:29.859468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.860127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.860600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.860730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.861422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:29.861727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.863489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.884130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1227/2490] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:29.889088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.898992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e8cb1-4489-4771-ab80-e8a0e018ce29 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:29.904047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:29.904772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:29.905044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:29.905325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:29.906069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:29.908000 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:29.944274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1264/2491] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:29.949563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:29.970837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f772713-2a13-4fdb-bb08-3362579e82c2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.059957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.060665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.061262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.061262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.062051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.064166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.086728 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1228/2492] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:29 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:30.089620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.104794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.106878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.108165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5f4d-1de4-4e3e-9c17-d605f07f5042 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.113932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.114623 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.114911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.115315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.116502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.118477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.141664 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1265/2493] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:30.150510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.182637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e403-79f1-47c5-a791-63285856993c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.188924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.190187 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.190771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.191253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.192560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.194944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.209048 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1266/2494] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:30.211045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.215856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.220430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0260e-86f1-4914-ad95-6f43128553b5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.226830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.227427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.227720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.228003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.230334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.236460 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e6e-4179-4902-b309-47b5f3aa6ba9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.242759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.244998 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.245831 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.246299 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] PATH_INFO: `/v3/users/b22129ca8c714db6ad44dee94e6099e9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.246770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.248388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22129ca8c714db6ad44dee94e6099e9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.248831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.251301 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.261535 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1229/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:30.262472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.274675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e568eef-e1b7-4e48-85d0-f7eec0230a2e service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.280688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.293558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.293752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.295022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] PATH_INFO: `/v3/projects/835616b5768e4aadba3997ed67e423b1/users/87fc34f23819416caa231a62c921bce8/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.326777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835616b5768e4aadba3997ed67e423b1, user_id=87fc34f23819416caa231a62c921bce8, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.329695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d30241-f47e-4c86-90a4-a5ccbff8c442 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.336638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:30.339213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22129ca8c714db6ad44dee94e6099e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.347939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1230/2496] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:30 2022] PUT /identity/v3/projects/835616b5768e4aadba3997ed67e423b1/users/87fc34f23819416caa231a62c921bce8/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:30.356735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22129ca8c714db6ad44dee94e6099e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.358297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.358815 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.359254 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.372349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22129ca8c714db6ad44dee94e6099e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.373094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22129ca8c714db6ad44dee94e6099e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.380758 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22129ca8c714db6ad44dee94e6099e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.388511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1267/2497] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:30 2022] DELETE /identity/v3/users/b22129ca8c714db6ad44dee94e6099e9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:30.394104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] MFA Rules not processed for user `87fc34f23819416caa231a62c921bce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:30.395631 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.418569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:30.485494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1231/2498] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:30.487846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.489455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.557056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353089-128a-4a61-b4ca-99cf35299a2b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.561764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.562427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.562714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.563263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.564085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.566159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.596985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1268/2499] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:30 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:30.598329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.598480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.599778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.621820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94906a-673e-414f-afee-33949e4e2d65 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.671716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761f196-76a3-46f6-90c7-9d8587264924 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.676424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.677119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.677505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.678336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.679159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.681083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.685974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.686829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.687110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.687388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] PATH_INFO: `/v3/projects/eb73ce3627964df4a085d7d9a81a9f59` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.689479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb73ce3627964df4a085d7d9a81a9f59)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.691760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.702042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1269/2500] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:30 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:30.706493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.716043 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555feb2f-6077-401f-9577-ebd162c13139 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.720763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.721440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.721726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.722008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.722728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.724417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.734812 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1270/2501] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:30.738896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.745984 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb73ce3627964df4a085d7d9a81a9f59', 'request_id': 'req-5a5a7a90-4222-4c50-8644-baa895068441'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.750335 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19db1-1bd0-47a1-8dc0-5ce61686066c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.754033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb73ce3627964df4a085d7d9a81a9f59', 'request_id': 'req-5a5a7a90-4222-4c50-8644-baa895068441'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.760889 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb73ce3627964df4a085d7d9a81a9f59', 'request_id': 'req-5a5a7a90-4222-4c50-8644-baa895068441'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.773123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] The token cache is being invalidate because project eb73ce3627964df4a085d7d9a81a9f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:30.773577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.774306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a5a7a90-4222-4c50-8644-baa895068441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:30.776011 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1232/2502] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:30 2022] DELETE /identity/v3/projects/eb73ce3627964df4a085d7d9a81a9f59 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:30.776566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:30.781086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b317a-d730-453b-bc46-5115554a467c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.781406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b317a-d730-453b-bc46-5115554a467c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.781691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b317a-d730-453b-bc46-5115554a467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.800084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c74b317a-d730-453b-bc46-5115554a467c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:30.801281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c74b317a-d730-453b-bc46-5115554a467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.814928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.815864 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.816142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.816414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] PATH_INFO: `/v3/projects/7e284107279440368080a5ae6578f80b/users/2dcd91f61ac44ac29f15709af603c393/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.832094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e284107279440368080a5ae6578f80b, user_id=2dcd91f61ac44ac29f15709af603c393, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.834830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.844002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:30.848989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1271/2503] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:30 2022] PUT /identity/v3/projects/7e284107279440368080a5ae6578f80b/users/2dcd91f61ac44ac29f15709af603c393/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:30.853373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.855152 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.864800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1233/2504] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:30.872450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.872769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.873043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.892101 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:30.893299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:30.918654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604e71d-913b-4e91-a731-b3bd13b133b6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.923312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.923976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.924262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.924539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.925260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.926882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.936675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1272/2505] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:30.941020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.951203 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e712-dc47-41be-86e7-5ffe67ba07a7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.954088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1234/2506] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:30.955855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:30.956724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:30.957007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:30.957282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] PATH_INFO: `/v3/projects/7e284107279440368080a5ae6578f80b/users/2dcd91f61ac44ac29f15709af603c393/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:30.958079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:30.959858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e284107279440368080a5ae6578f80b, user_id=2dcd91f61ac44ac29f15709af603c393, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:30.962570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bc9c7-8b41-4c92-b38b-29ebbd90413b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:30.968071 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939dcc5-bddc-47ce-ab48-e79f43e7f431 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:30.972989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1273/2507] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:30 2022] PUT /identity/v3/projects/7e284107279440368080a5ae6578f80b/users/2dcd91f61ac44ac29f15709af603c393/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:31.028893 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.029506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.029813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.030062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.030791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:31.031097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.032713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.037765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.038122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.038465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.045994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1235/2508] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:39:31.050118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.060098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23571af5-3984-4754-880a-9246eb127811 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.065004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.065672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.065956 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.066242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.066515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] MFA Rules not processed for user `2dcd91f61ac44ac29f15709af603c393`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:31.066900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.067714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.069290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.096900 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1236/2509] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:31.100867 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.144695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1274/2510] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:31.156245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.157685 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.159499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:31.164079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630b6d-3d67-4d90-b500-5117568b8707 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.170311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.171010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.171298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.171575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.172267 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.174272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.180228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:31.196259 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1237/2511] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:31.200552 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.211132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db397fda-4c51-46b5-a772-910eb5d2fad1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.216078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.216727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.217009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.217282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.218041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.219608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.223841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06dd08-8e0a-440d-be87-a7a66b32ec91 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.230331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1238/2512] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:31.234115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.243794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b02a-b9a3-46be-b580-9adc6feabae2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.248511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.249155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.249561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.249808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.250521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.252245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.262955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1239/2513] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:31.267048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.277178 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9732be3-5cf5-4843-a793-a94678863c8b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.281320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.281945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.282181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.282315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.282489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.283057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.283335 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.283810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] PATH_INFO: `/v3/projects/70424b4a998a46a0a9e70d76e0d4e36d/users/467530cfb4c8434c88c705f81dc7f845/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.298986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70424b4a998a46a0a9e70d76e0d4e36d, user_id=467530cfb4c8434c88c705f81dc7f845, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.301645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ee2b-888c-412e-8ce6-5677e5b656e2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.316307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1240/2514] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:31 2022] PUT /identity/v3/projects/70424b4a998a46a0a9e70d76e0d4e36d/users/467530cfb4c8434c88c705f81dc7f845/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:31.327376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.327676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.328017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.347065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.348514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.356312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] MFA Rules not processed for user `467530cfb4c8434c88c705f81dc7f845`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:31.357543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.359090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1275/2515] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:31.450634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.461237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b4d2-499a-4701-89f4-f83fdffc79cd service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.514359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.515092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.515379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.516025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.517910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.559859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:31.562616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1241/2516] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:31.574033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.575388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:31.582332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.583772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.593044 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1276/2517] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:31.658955 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101a903-accd-4afd-8398-a5a30f56cdb9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.666754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.668268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.668685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.668971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.677275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.679024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90369b95-3e9b-46cc-9172-263c08f5912c service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.690190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1242/2518] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:31.859615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:31.928563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438631-e218-422f-b3ea-193a9efb2ad3 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:31.934621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:31.935731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:31.936092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:31.936372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] PATH_INFO: `/v3/users/87fc34f23819416caa231a62c921bce8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:31.937597 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87fc34f23819416caa231a62c921bce8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:31.939766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:31.995838 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87fc34f23819416caa231a62c921bce8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.010722 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87fc34f23819416caa231a62c921bce8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.029725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87fc34f23819416caa231a62c921bce8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.030540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87fc34f23819416caa231a62c921bce8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.038746 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87fc34f23819416caa231a62c921bce8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.050789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1277/2519] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:31 2022] DELETE /identity/v3/users/87fc34f23819416caa231a62c921bce8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:32.056487 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.056963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.057218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.075765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:32.077000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.086829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:32.136892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1243/2520] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:32.147947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.149398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.207168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411d064-540f-40de-be54-1ac28b9be2f5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.210902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.211550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.211836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.212120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.221211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.222659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.227636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:32.233493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1278/2521] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:32.504192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.505791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:32.518293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534178f7-e9ba-4e9b-b3ba-fa20f03cf454 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.523226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.524094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.524381 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.524656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] PATH_INFO: `/v3/projects/835616b5768e4aadba3997ed67e423b1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.525862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835616b5768e4aadba3997ed67e423b1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.527756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.575707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835616b5768e4aadba3997ed67e423b1', 'request_id': 'req-9a1b432d-8198-47da-9a25-f9e53d16b873'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.582720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835616b5768e4aadba3997ed67e423b1', 'request_id': 'req-9a1b432d-8198-47da-9a25-f9e53d16b873'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.589287 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835616b5768e4aadba3997ed67e423b1', 'request_id': 'req-9a1b432d-8198-47da-9a25-f9e53d16b873'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.601001 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] The token cache is being invalidate because project 835616b5768e4aadba3997ed67e423b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:32.601289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.602024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.603559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1244/2522] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:32 2022] DELETE /identity/v3/projects/835616b5768e4aadba3997ed67e423b1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:32.607572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:32.618190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c02f4-3322-4608-9824-8217b0367778 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:32.674899 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:32.675801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:32.676081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:32.676353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] PATH_INFO: `/v3/users/2dcd91f61ac44ac29f15709af603c393` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:32.677591 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcd91f61ac44ac29f15709af603c393)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:32.680084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:32.726433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcd91f61ac44ac29f15709af603c393'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.733702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcd91f61ac44ac29f15709af603c393'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.745569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcd91f61ac44ac29f15709af603c393'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.746216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcd91f61ac44ac29f15709af603c393'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.753031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3f7cb728-b4df-4bfb-b7d5-fa10d3e7ad27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcd91f61ac44ac29f15709af603c393'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:32.759936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1279/2523] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:32 2022] DELETE /identity/v3/users/2dcd91f61ac44ac29f15709af603c393 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:33.024945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.026488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.042429 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b432d-8198-47da-9a25-f9e53d16b873 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.100344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.101212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.101644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.101813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] PATH_INFO: `/v3/projects/7e284107279440368080a5ae6578f80b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.103002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e284107279440368080a5ae6578f80b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.104934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.147530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:33.154908 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e284107279440368080a5ae6578f80b', 'request_id': 'req-f1997bbf-597b-4d97-9f85-42a7e422260f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.163757 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e284107279440368080a5ae6578f80b', 'request_id': 'req-f1997bbf-597b-4d97-9f85-42a7e422260f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.169465 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e284107279440368080a5ae6578f80b', 'request_id': 'req-f1997bbf-597b-4d97-9f85-42a7e422260f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.181010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] The token cache is being invalidate because project 7e284107279440368080a5ae6578f80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:33.181301 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.182005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f1997bbf-597b-4d97-9f85-42a7e422260f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:33.183530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1245/2524] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:33 2022] DELETE /identity/v3/projects/7e284107279440368080a5ae6578f80b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:33.188662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.188960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.189251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.208359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.209691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.281418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1280/2525] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:33.283208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.283496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.283772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.303298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.304581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.313312 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1246/2526] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:33.317937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.328785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7d45b-fc24-4143-a528-7a721b0f0294 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.334944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.335616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.335941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.336271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.337006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:33.337317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.338985 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.351556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1281/2527] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:33.355715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.364918 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daca9c-a600-4e54-a5e6-84a8502b0ce2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.370202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.370847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.371132 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.371408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.372122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.374044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.398632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1247/2528] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:33.402636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.413220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cef55-79d6-4c75-8dbd-ebc9987102d2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.416950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:33.471384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.472056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.472336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.472610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.473287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.475766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.497451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1282/2529] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:33.501418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.514228 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be710c4-ac1f-40e9-9b2d-3a2a8a4c96e9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.519756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.520483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.520844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.521125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.521940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.523635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.534977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1248/2530] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:33.539374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.549709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51672-281b-4fb7-a51b-35b46f37a24f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.555598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.556280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.556580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.556905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.557613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.559212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aceada0-b4f0-4862-90c3-d889a56aab69 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.570330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1283/2531] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:33.575056 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.586593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caa826-9044-449e-8448-4b3f745cc5e4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.591346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.592293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.592574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.592849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] PATH_INFO: `/v3/projects/2b14263da0014048894dd7b11686a114/users/47900371276e409f985a7c43d8f9224d/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.618252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b14263da0014048894dd7b11686a114, user_id=47900371276e409f985a7c43d8f9224d, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.621839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.638665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1249/2532] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:33 2022] PUT /identity/v3/projects/2b14263da0014048894dd7b11686a114/users/47900371276e409f985a7c43d8f9224d/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:33.648952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.649479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.649607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.675780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] MFA Rules not processed for user `47900371276e409f985a7c43d8f9224d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:33.677022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.743184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1284/2533] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:33.753965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:33.755789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:33.814439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dcce3-1da6-4ec3-a552-a05f5dd42d74 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:33.818128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:33.819265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:33.819551 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:33.819829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:33.828480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:33.829978 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951705-e03a-433d-8449-b5483a98bb84 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:33.839860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1250/2534] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:34.387084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.387084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.450692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f07c8-965a-4602-a38e-1dca057ced07 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.455230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.456103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.456384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.456657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] PATH_INFO: `/v3/users/467530cfb4c8434c88c705f81dc7f845` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.457854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=467530cfb4c8434c88c705f81dc7f845)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.459745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.475133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:34.507281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467530cfb4c8434c88c705f81dc7f845'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:34.514896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467530cfb4c8434c88c705f81dc7f845'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:34.527119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467530cfb4c8434c88c705f81dc7f845'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:34.527738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467530cfb4c8434c88c705f81dc7f845'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:34.534598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467530cfb4c8434c88c705f81dc7f845'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:34.541934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1285/2535] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:34 2022] DELETE /identity/v3/users/467530cfb4c8434c88c705f81dc7f845 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:34.546452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.546962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.547079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.566064 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:34.567260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.625067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1251/2536] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:34.635119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.636520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:34.695556 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ed75-e6fa-4b55-b744-98006f7cd5b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:34.699298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:34.699942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:34.700222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:34.700494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:34.710112 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:34.711538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc223f-7ade-4b00-b347-26fe769be888 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:34.721062 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1286/2537] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:34.989197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:34.990672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.002492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029338d-ebce-4fb4-aac9-504913df5adc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.007892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.008922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.009171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.009479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] PATH_INFO: `/v3/projects/70424b4a998a46a0a9e70d76e0d4e36d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.011852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70424b4a998a46a0a9e70d76e0d4e36d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.013780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.017073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.063949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70424b4a998a46a0a9e70d76e0d4e36d', 'request_id': 'req-1730d1c5-c212-4415-9510-19d8368b9205'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:35.071088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70424b4a998a46a0a9e70d76e0d4e36d', 'request_id': 'req-1730d1c5-c212-4415-9510-19d8368b9205'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:35.076871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70424b4a998a46a0a9e70d76e0d4e36d', 'request_id': 'req-1730d1c5-c212-4415-9510-19d8368b9205'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:35.088164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] The token cache is being invalidate because project 70424b4a998a46a0a9e70d76e0d4e36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:35.088448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:35.089643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1730d1c5-c212-4415-9510-19d8368b9205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:35.091154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1252/2538] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:34 2022] DELETE /identity/v3/projects/70424b4a998a46a0a9e70d76e0d4e36d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:35.096012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.096300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.096599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.114457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:35.116155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.176228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1287/2539] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.184484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.184803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.185085 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.204446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:35.205690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.213708 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1253/2540] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.217947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.231596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9905d56-3318-4132-af0f-46279f1eab50 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.233789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.234461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.234741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.235012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.235773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:35.236079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.237760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.251909 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1288/2541] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:35.256545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.266355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a10202-ccdd-4a4f-a490-dd262e59be26 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.272114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.272842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.273411 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.273953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.274721 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.276822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.303226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1254/2542] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:35.307785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.319681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf579a1-b96d-440e-b3d5-a69acd38afbe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.344418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.385045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.385774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.386059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.386333 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.387066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.388989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.415777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1289/2543] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:35.422091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.433168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b0af8-06bc-42b8-bc51-662a3e7fb9db admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.437727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.438636 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.438917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.439197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.439911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.441599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.452971 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1255/2544] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:35.457089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.467631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa11a-9c9a-4153-bd2c-470a7dbf0226 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.474003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.475173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.475457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.475735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.476464 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.478160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb238b-3515-4e7a-8375-16f671d5b687 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.488844 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1290/2545] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:35.493253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.504309 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118dfd0-9b41-419d-b305-597c2678c0a5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.509906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.510860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.511272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.511740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] PATH_INFO: `/v3/projects/77ba8dd6e49c4ff590008563414483d8/users/e1546f52d8b6415997e7910201db1a92/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.528356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ba8dd6e49c4ff590008563414483d8, user_id=e1546f52d8b6415997e7910201db1a92, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.531240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.547511 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1256/2546] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:35 2022] PUT /identity/v3/projects/77ba8dd6e49c4ff590008563414483d8/users/e1546f52d8b6415997e7910201db1a92/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:35.558260 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.558689 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.559167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.590081 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] MFA Rules not processed for user `e1546f52d8b6415997e7910201db1a92`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:35.591519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:35.666584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1291/2547] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:35.671234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.739699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d312fa-a47f-499c-bd86-1b01c7c1eb81 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.743475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.744131 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.744414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.744698 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.745403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.747274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.775406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1257/2548] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:35.779776 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.790762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d481b-1be0-4bff-9571-6b8ca52e2db2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.852162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.852850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.853136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.853450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.854139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.856232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.875502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1292/2549] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:35 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:35.879611 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.890928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273674cb-c0e2-469f-b3c6-3f2ea6c9e2d7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.895592 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.896245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.896547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.896940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.897691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.899272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.909675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1258/2550] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:35.913684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.923560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2d294-b5fe-4125-9cd6-dee416aaded8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:35.928687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:35.929586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:35.929875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:35.930180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] PATH_INFO: `/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7/users/60cdaa9d97384eb0843c6b9cf755d91d/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:35.944862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=180a6ee1350743f0aa73e5e55dcab8d7, user_id=60cdaa9d97384eb0843c6b9cf755d91d, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:35.947594 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:35.955140 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.962173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1293/2551] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:35 2022] PUT /identity/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7/users/60cdaa9d97384eb0843c6b9cf755d91d/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:35.967236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:35.973161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:35.980005 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a34978-91c4-4a0c-9cb8-b7eea6ac31f8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.036745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.037931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.038227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.038527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.039265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.040881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e8223-8225-4f3f-986d-70f2b78dc633 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.050763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1259/2552] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:36.055038 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.068010 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cabad7-189b-48f7-b79b-abeb959f9a3f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.076880 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.079367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.079991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.080641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] PATH_INFO: `/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7/users/60cdaa9d97384eb0843c6b9cf755d91d/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.084904 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=180a6ee1350743f0aa73e5e55dcab8d7, user_id=60cdaa9d97384eb0843c6b9cf755d91d, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.087770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.099678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1294/2553] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:36 2022] PUT /identity/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7/users/60cdaa9d97384eb0843c6b9cf755d91d/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:36.111286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.111582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.111869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.139124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] MFA Rules not processed for user `60cdaa9d97384eb0843c6b9cf755d91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:36.140331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.208110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1260/2554] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:36.219876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.221624 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.280401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289ec7f-dee9-4aaf-aa7e-1907d5e31dcb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.284003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.284644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.284921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.285195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.294613 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.295955 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.305728 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1295/2555] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:36.375049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.385501 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d8ad7-96a1-4377-a91c-5318bf28d7fe None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.389173 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.389847 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.390133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.390416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.448540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.450004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322142ee-9874-49e5-96dc-713a15ab3d5b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.458962 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1261/2556] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:36.870355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:36.871755 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:36.935395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45f930-fe53-48b8-b8f2-2eb384a209c4 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:36.940200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:36.941067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:36.941372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:36.941660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] PATH_INFO: `/v3/users/47900371276e409f985a7c43d8f9224d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:36.942846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47900371276e409f985a7c43d8f9224d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:36.944756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:36.988232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47900371276e409f985a7c43d8f9224d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:36.995953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47900371276e409f985a7c43d8f9224d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.007442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47900371276e409f985a7c43d8f9224d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.008097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47900371276e409f985a7c43d8f9224d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.014988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47900371276e409f985a7c43d8f9224d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.020775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1296/2557] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:36 2022] DELETE /identity/v3/users/47900371276e409f985a7c43d8f9224d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:37.025284 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.025794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.025894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.043714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:37.044910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.062058 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:37.104778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1262/2558] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:37.114621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.116024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.125567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:37.175258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdc87b-c640-4307-b437-f38652dfda54 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.179441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.180082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.180363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.180638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.190027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.191442 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d438f-9e5e-4d84-a7eb-9b4449c608cd service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.200770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1297/2559] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:37.481711 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.483585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.500540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480040f4-714c-4dfd-a138-1ad1bca8543d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.505399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.506424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.506805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.507197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] PATH_INFO: `/v3/projects/2b14263da0014048894dd7b11686a114` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.508477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b14263da0014048894dd7b11686a114)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.510490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.580643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b14263da0014048894dd7b11686a114', 'request_id': 'req-3e872f3b-67fe-4c7f-9a2c-262339462ba1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.589858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b14263da0014048894dd7b11686a114', 'request_id': 'req-3e872f3b-67fe-4c7f-9a2c-262339462ba1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.597183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b14263da0014048894dd7b11686a114', 'request_id': 'req-3e872f3b-67fe-4c7f-9a2c-262339462ba1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.611904 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] The token cache is being invalidate because project 2b14263da0014048894dd7b11686a114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:37.612266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.613052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e872f3b-67fe-4c7f-9a2c-262339462ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:37.615438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1263/2560] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:37 2022] DELETE /identity/v3/projects/2b14263da0014048894dd7b11686a114 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:37.620470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.620945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.621457 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.641802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:37.643929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.715490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1298/2561] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:37.722936 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.723310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.723591 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.747794 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:37.749014 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:37.760475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1264/2562] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:37.765147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.776032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7192c-76c7-4f22-98ba-807d3bd8e2f7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.783595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.784269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.784555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.784835 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.785606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:37.785924 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.787679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.801467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1299/2563] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:37.807882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.817783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdda8-f386-45e8-84a1-7ddffcf43056 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.823201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.823858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.824140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.824440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.825130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.827039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.855600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1265/2564] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:37.859529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.870606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f26749-9c30-4f8a-bf06-4d37ca1523db admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.930026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.930702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.930986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.931267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.931971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.933883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.955535 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1300/2565] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:37 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:37.959999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:37.974195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bbd84-d7f7-4845-97c9-94b760698095 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:37.978975 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:37.979714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:37.980148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:37.980484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:37.981843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:37.983482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:37.994514 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1266/2566] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:37.998461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.008332 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020c0f2-05a3-42c9-8914-4e0bb5bd90f7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.012965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.013652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.013946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.014232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.014961 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.016546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e9a4a-3b5b-4d85-8f24-a85dc688eae5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.031217 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1301/2567] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:38.032339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.043111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37689689-90a0-450b-af7a-949d6a3fc187 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.048091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.049088 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.049236 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.049609 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] PATH_INFO: `/v3/projects/f8ff4496a9884d6ba5f53803d4aa475b/users/294ef0f0e5e04c4193eba83a679afed1/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.065577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff4496a9884d6ba5f53803d4aa475b, user_id=294ef0f0e5e04c4193eba83a679afed1, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.068247 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.086546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1267/2568] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:38 2022] PUT /identity/v3/projects/f8ff4496a9884d6ba5f53803d4aa475b/users/294ef0f0e5e04c4193eba83a679afed1/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:38.091599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84343d-90aa-4014-9fa2-2e66ee36dea7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.091897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84343d-90aa-4014-9fa2-2e66ee36dea7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.092185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84343d-90aa-4014-9fa2-2e66ee36dea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.110711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6e84343d-90aa-4014-9fa2-2e66ee36dea7 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:38.111968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e84343d-90aa-4014-9fa2-2e66ee36dea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:38.121147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:38.176120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1302/2569] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:38.185799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.187635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:38.233070 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:38.246726 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba98a-bfd7-4315-ac0d-dd7cf49dd6a1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:38.250483 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:38.251083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.251370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.251659 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.261387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:38.262807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:38.272187 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1268/2570] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:38.864416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:38.864752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:38.865580 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:38.892768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] MFA Rules not processed for user `294ef0f0e5e04c4193eba83a679afed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:38.894012 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:38.962902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1303/2571] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:38.973970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:38.976087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.036238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a7dd-3ba8-4129-aaa4-893788ad482f service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.040550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.041198 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.041519 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.041901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.050724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.052356 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60f31-266f-4628-9fcf-891b980c5835 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.061831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1269/2572] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:39.205619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.206982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.221704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858296c4-1675-42cf-82cb-365b84e47796 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.227413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.228370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.228662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.228942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] PATH_INFO: `/v3/users/e1546f52d8b6415997e7910201db1a92` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.230199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1546f52d8b6415997e7910201db1a92)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.232128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.280183 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1546f52d8b6415997e7910201db1a92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.288968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1546f52d8b6415997e7910201db1a92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.301784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1546f52d8b6415997e7910201db1a92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.302633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1546f52d8b6415997e7910201db1a92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.311001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1546f52d8b6415997e7910201db1a92'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.320821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1304/2573] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:39 2022] DELETE /identity/v3/users/e1546f52d8b6415997e7910201db1a92 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:39.322949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.323522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.323899 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.342865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:39.344220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.407803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1270/2574] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:39.418156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.419652 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.440200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:39.480490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1ac10-df71-4424-9bd8-05d577d9699f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.484225 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.484887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.485169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.485486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.497176 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.498680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.509905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1305/2575] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:39.781547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.782987 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:39.796243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801943c-5bbb-4836-ab1f-92d1ce0c1202 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.801416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.802405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.803268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.803550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] PATH_INFO: `/v3/projects/77ba8dd6e49c4ff590008563414483d8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.804792 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ba8dd6e49c4ff590008563414483d8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.806963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:39.855022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ba8dd6e49c4ff590008563414483d8', 'request_id': 'req-ad3bb97d-5480-438e-9e15-6e5733c138c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.862268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ba8dd6e49c4ff590008563414483d8', 'request_id': 'req-ad3bb97d-5480-438e-9e15-6e5733c138c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.868603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ba8dd6e49c4ff590008563414483d8', 'request_id': 'req-ad3bb97d-5480-438e-9e15-6e5733c138c4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.879484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] The token cache is being invalidate because project 77ba8dd6e49c4ff590008563414483d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:39.879762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.880393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ad3bb97d-5480-438e-9e15-6e5733c138c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:39.881949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1271/2576] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:39 2022] DELETE /identity/v3/projects/77ba8dd6e49c4ff590008563414483d8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:39.885996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:39.895706 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260427d5-98c4-409a-bbde-a852b0db557c service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:39.953396 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:39.954398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:39.954708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:39.955018 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] PATH_INFO: `/v3/users/60cdaa9d97384eb0843c6b9cf755d91d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:39.956228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cdaa9d97384eb0843c6b9cf755d91d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:39.958159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.002749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cdaa9d97384eb0843c6b9cf755d91d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.010353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cdaa9d97384eb0843c6b9cf755d91d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.022283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cdaa9d97384eb0843c6b9cf755d91d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.022913 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cdaa9d97384eb0843c6b9cf755d91d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.029810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cdaa9d97384eb0843c6b9cf755d91d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.035546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1306/2577] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:39 2022] DELETE /identity/v3/users/60cdaa9d97384eb0843c6b9cf755d91d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:40.437019 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b678-e5d6-4347-99bc-d98d605abfa0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.437498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b678-e5d6-4347-99bc-d98d605abfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.437775 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b678-e5d6-4347-99bc-d98d605abfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.457939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6bd7b678-e5d6-4347-99bc-d98d605abfa0 None None] MFA Rules not processed for user `294ef0f0e5e04c4193eba83a679afed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:40.459844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6bd7b678-e5d6-4347-99bc-d98d605abfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.465048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:40.517448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1272/2578] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:40.571390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.572821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.626651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:40.651915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59883f4-f4c4-4f7b-a61d-7e0599c689d8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:40.657068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:40.657958 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.658242 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.658520 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] PATH_INFO: `/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.659710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180a6ee1350743f0aa73e5e55dcab8d7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:40.661658 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.710792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180a6ee1350743f0aa73e5e55dcab8d7', 'request_id': 'req-a32f00c9-dc57-48db-bdd0-12f33fbb9273'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.718885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180a6ee1350743f0aa73e5e55dcab8d7', 'request_id': 'req-a32f00c9-dc57-48db-bdd0-12f33fbb9273'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.725702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180a6ee1350743f0aa73e5e55dcab8d7', 'request_id': 'req-a32f00c9-dc57-48db-bdd0-12f33fbb9273'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.737056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] The token cache is being invalidate because project 180a6ee1350743f0aa73e5e55dcab8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:40.737376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.738115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:40.739662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1307/2579] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:40 2022] DELETE /identity/v3/projects/180a6ee1350743f0aa73e5e55dcab8d7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:40.744729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a448-d877-4fce-94ee-129e1285ec07 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.745096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a448-d877-4fce-94ee-129e1285ec07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.745479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a448-d877-4fce-94ee-129e1285ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.762186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.763616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.764449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3080a448-d877-4fce-94ee-129e1285ec07 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:40.765752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3080a448-d877-4fce-94ee-129e1285ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.826420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f00c9-dc57-48db-bdd0-12f33fbb9273 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:40.828222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1273/2580] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:40.830186 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:40.830839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.831127 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.831410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.833213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.835696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.836124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.836408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.843537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:40.845011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.853898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1308/2581] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:39:40.855832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:40.857069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:40.865084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1274/2582] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:40.869953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.881463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758fb8d-cdd0-44f5-9352-0635382066b9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:40.886762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:40.887417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.887704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.887987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.888716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:40.889029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:40.890661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.902681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1309/2583] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:40.907057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.916937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa28b8-150e-4e47-989f-e82037dbfc0d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:40.921706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:40.922355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:40.922637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:40.922916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:40.923605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:40.925506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:40.951648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1275/2584] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:40.955820 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:40.966657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe06a4-3b01-4a5b-9d7c-737aee546cf4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.024951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.025678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.025972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.026252 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.027130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.028888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.035770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:41.051274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1310/2585] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:40 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:41.055546 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.066776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b933ee-45af-4929-8666-55dc66102be5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.072150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.072856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.073184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.073509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.074244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.075871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.086720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1276/2586] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:41.091137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.101223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2495d4a-15ed-4f1e-91f2-11e37258d5c8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.105921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.106644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.106992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.107354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.108141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.109960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea552c4-df15-4693-b554-e49416ed4614 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.121016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1311/2587] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:41.125457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.135896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c3a-b8a6-4552-bd81-e1f54fad2583 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.140733 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.141735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.142114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.142479 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] PATH_INFO: `/v3/projects/58ea641a459b422e8f154d8d93567dfb/users/d28936396d584bfe8a099649e16108b4/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.165608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ea641a459b422e8f154d8d93567dfb, user_id=d28936396d584bfe8a099649e16108b4, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.168508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.196404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1277/2588] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:41 2022] PUT /identity/v3/projects/58ea641a459b422e8f154d8d93567dfb/users/d28936396d584bfe8a099649e16108b4/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:41.206897 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.207195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.207474 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.254351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] MFA Rules not processed for user `d28936396d584bfe8a099649e16108b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:41.255610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:41.323481 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1312/2589] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:41.328050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.383949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:41.395738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad60e65-4cde-45ce-9f76-31c8f6cb0b97 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.400571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.401234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.401562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.404948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.405887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.407967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.439662 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1278/2590] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:41 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:41.440900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.452994 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed5291-a238-4414-9fab-4c1776ba8512 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.511991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.512676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.512984 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.513288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.514611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.516363 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.542156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1313/2591] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:41 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:41.546422 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.557694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6afdd-f61c-4637-8402-c3610532061d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.564112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.565152 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.565480 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.565776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.566580 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.568859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.579894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1279/2592] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:41.584800 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.594841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55af5f-0380-4eba-968e-1ef5baf89c19 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.599931 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.600809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.601088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.601513 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] PATH_INFO: `/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c/users/8ad468eb3b86456096e9ab2076bb2d34/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.619310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9bc542d1244431b9fd01fb7ea6b96c, user_id=8ad468eb3b86456096e9ab2076bb2d34, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.621972 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.635976 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1314/2593] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:41 2022] PUT /identity/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c/users/8ad468eb3b86456096e9ab2076bb2d34/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:41.640234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.650822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169a60d-376e-44e5-bb67-de8e5f7610b0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.707372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.708047 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.708331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.708602 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.709883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.711468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf58ae-050d-4028-9760-20e5c3deb8cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.721579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1280/2594] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:41.726473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.736784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc419899-fab0-4062-9796-2de96a424431 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.741541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.742459 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.742712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.742991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] PATH_INFO: `/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c/users/8ad468eb3b86456096e9ab2076bb2d34/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.745040 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9bc542d1244431b9fd01fb7ea6b96c, user_id=8ad468eb3b86456096e9ab2076bb2d34, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.747760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.759628 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1315/2595] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:41 2022] PUT /identity/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c/users/8ad468eb3b86456096e9ab2076bb2d34/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:41.771874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.772332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.772588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.802089 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] MFA Rules not processed for user `8ad468eb3b86456096e9ab2076bb2d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:41.803466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:41.872613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1281/2596] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:41.883466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:41.885644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:41.909237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:41.946106 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57613-fb4c-4bd9-93d2-12676fc5a8c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:41.949726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:41.950365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:41.950647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:41.950949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:41.959775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:41.961184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e1b3c-b021-45ec-8db8-f3279812fd70 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:41.970518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1316/2597] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:41.997988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:42.049649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:42.066199 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895211a-3bc8-4548-95d7-54eef2d0c419 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:42.071343 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:42.072489 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.072617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.072883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] PATH_INFO: `/v3/users/d28936396d584bfe8a099649e16108b4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.074299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28936396d584bfe8a099649e16108b4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:42.076277 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:42.130672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28936396d584bfe8a099649e16108b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.139807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28936396d584bfe8a099649e16108b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.140503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28936396d584bfe8a099649e16108b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.154459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28936396d584bfe8a099649e16108b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.161558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28936396d584bfe8a099649e16108b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.169032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1282/2598] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:41 2022] DELETE /identity/v3/users/d28936396d584bfe8a099649e16108b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:42.173778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.174085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.174375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.195687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:42.196933 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:42.297906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1317/2599] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:42.308211 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:42.311606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:42.378961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a2ac-fb92-434c-9584-e611a03ff636 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:42.383360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:42.384081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.384560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.384960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.397035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:42.398508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:42.413444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1283/2600] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:42.688618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:42.690575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:42.713836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800ea4c-aa45-43f8-862e-a617e310df19 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:42.720028 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:42.720900 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.721179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.721517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] PATH_INFO: `/v3/projects/58ea641a459b422e8f154d8d93567dfb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.722869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ea641a459b422e8f154d8d93567dfb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:42.724855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:42.772229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ea641a459b422e8f154d8d93567dfb', 'request_id': 'req-774fc34f-8f7f-42a7-954a-2746dbee5c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.780721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ea641a459b422e8f154d8d93567dfb', 'request_id': 'req-774fc34f-8f7f-42a7-954a-2746dbee5c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.786602 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ea641a459b422e8f154d8d93567dfb', 'request_id': 'req-774fc34f-8f7f-42a7-954a-2746dbee5c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.797663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] The token cache is being invalidate because project 58ea641a459b422e8f154d8d93567dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:42.797957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.798606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.800448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1318/2601] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:42 2022] DELETE /identity/v3/projects/58ea641a459b422e8f154d8d93567dfb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:42.804996 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:42.815075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d5dd6-2159-4e87-893d-b8d014768a19 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:42.877010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:42.877934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:42.878219 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:42.878500 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] PATH_INFO: `/v3/users/8ad468eb3b86456096e9ab2076bb2d34` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:42.879729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad468eb3b86456096e9ab2076bb2d34)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:42.881729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:42.895296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:42.933585 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad468eb3b86456096e9ab2076bb2d34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.941952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad468eb3b86456096e9ab2076bb2d34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.942627 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad468eb3b86456096e9ab2076bb2d34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.956512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad468eb3b86456096e9ab2076bb2d34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.963941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6cd7ad46-c92d-487b-b7c0-1982327ef436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad468eb3b86456096e9ab2076bb2d34'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:42.970430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1284/2602] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:42 2022] DELETE /identity/v3/users/8ad468eb3b86456096e9ab2076bb2d34 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:43.243554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.245101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:43.260711 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fc34f-8f7f-42a7-954a-2746dbee5c8d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.315751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:43.332001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.332873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.333156 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.333497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] PATH_INFO: `/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.335318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9bc542d1244431b9fd01fb7ea6b96c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.337370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.388511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9bc542d1244431b9fd01fb7ea6b96c', 'request_id': 'req-2df7f1f7-8d74-4023-86c0-b052215a8576'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:43.397577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9bc542d1244431b9fd01fb7ea6b96c', 'request_id': 'req-2df7f1f7-8d74-4023-86c0-b052215a8576'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:43.407937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9bc542d1244431b9fd01fb7ea6b96c', 'request_id': 'req-2df7f1f7-8d74-4023-86c0-b052215a8576'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:43.428244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] The token cache is being invalidate because project 6a9bc542d1244431b9fd01fb7ea6b96c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:43.428537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:43.429202 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2df7f1f7-8d74-4023-86c0-b052215a8576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:43.435536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1319/2603] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:43 2022] DELETE /identity/v3/projects/6a9bc542d1244431b9fd01fb7ea6b96c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:43.437482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.437837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.438799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.460374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:43.461597 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:43.532526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1285/2604] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:39:43.540391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.541205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.541205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.564881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:43.566155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:43.588069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1320/2605] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:43.588238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.596285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90be1-f86c-43a1-adae-22ff9005024f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.601912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.602586 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.602869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.603458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.604694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:43.604694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.606118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.620856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1286/2606] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:43.625267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.637060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f821e4e-7964-44ca-a7f7-15d3abc1708f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.642725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.643582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.644241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.644241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.645102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.647147 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.676412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1321/2607] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:43 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:43.677879 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.687979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7c229-cc60-45b0-bb2e-361f8eaf90f8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.745084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.745781 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.746066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.746342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.747023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.749040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.769825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1287/2608] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:43 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:43.774034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.785435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da689aa-1e1a-430c-96e1-28c56bd94f44 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.791001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.791668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.791954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.792232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.792962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.794586 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.805177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1322/2609] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:43.809144 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.820007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839962-1f9f-4c7a-8aa5-0afffb1c3a19 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.824595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.825256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.825572 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.825855 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.826595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.828170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2875f41-14e8-4a53-89ac-58d9afa1ea01 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.829691 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:43.844509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1288/2610] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:39:43.845672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:43.856067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6951a-7626-4b55-9ec0-f607e0a6c2ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:43.861969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:43.862920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.863206 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.863507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] PATH_INFO: `/v3/projects/b42097a844764c388a30caf0bd86cb4b/users/ffbee363db704dc3a6e68961bf62260a/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.878605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42097a844764c388a30caf0bd86cb4b, user_id=ffbee363db704dc3a6e68961bf62260a, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:43.881794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:43.894668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1323/2611] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:43 2022] PUT /identity/v3/projects/b42097a844764c388a30caf0bd86cb4b/users/ffbee363db704dc3a6e68961bf62260a/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:43.906457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:43.906750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:43.907026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:43.935261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] MFA Rules not processed for user `ffbee363db704dc3a6e68961bf62260a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:43.936405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.003529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1289/2612] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:44.007934 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.070818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ef729-f4f7-41f4-b4d2-c2581df22fa3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.075454 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.076113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.076395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.076675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.077378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.079259 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.099390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:44.108802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1324/2613] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:44.112644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.123568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996740d6-3130-4a9c-803f-7f9ffb56d99c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.180398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.181096 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.181418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.181701 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.182390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.184303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.204793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1290/2614] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:44.208714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.218915 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99b0b7-deec-45e0-a641-6b73667cc7ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.223490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.224149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.224434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.224713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.225504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.227096 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.237612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1325/2615] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:44.242061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.252506 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1bf7-e7ac-4115-96ac-1859632e96b0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.258594 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.259255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.259541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.259849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.260581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.262238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306df59a-2188-4d1c-91c0-902a7f99cc22 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.272279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1291/2616] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:44.276588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.287136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6665b39-f035-43fe-8579-1095ea611828 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.291845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.292731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.293017 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.293294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] PATH_INFO: `/v3/projects/178915e320884ce79663c8f0466d1560/users/fc8bab25a65240839f10b739b52154dc/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.308411 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178915e320884ce79663c8f0466d1560, user_id=fc8bab25a65240839f10b739b52154dc, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.311094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.324380 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1326/2617] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:44 2022] PUT /identity/v3/projects/178915e320884ce79663c8f0466d1560/users/fc8bab25a65240839f10b739b52154dc/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:44.334471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.334762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.335072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.363470 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] MFA Rules not processed for user `fc8bab25a65240839f10b739b52154dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:44.364700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.418577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:44.429690 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1292/2618] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:44.434393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.494885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8642-fd59-4e8c-9372-a6ed46e7c121 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.499435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.500088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.500374 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.500655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.501343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.503256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.528243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1327/2619] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:44.532357 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.542938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f862a-ec79-4b92-ba6c-afddb4f5701c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.598158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.598823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.599104 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.599868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.600542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.602455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.623030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1293/2620] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:44.627075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.637482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddd92c-39f1-4297-b40e-41ccdc5889a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.642136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.642792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.643071 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.643343 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.644101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.645731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.656661 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1328/2621] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:44.660868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.671359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade116-dfad-4b31-9551-5b3765c70a4f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.676091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.677046 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.677239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.677650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] PATH_INFO: `/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440/users/cf47305a96664a67a2df9df23c08eff4/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.692644 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a23cb1a0c6147e6b0a156cbc1c30440, user_id=cf47305a96664a67a2df9df23c08eff4, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.695308 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.708432 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1294/2622] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:44 2022] PUT /identity/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440/users/cf47305a96664a67a2df9df23c08eff4/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:44.713049 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.723165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194bbc7-05e0-43a9-8625-07f49ea089af admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.787557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.788222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.788504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.788778 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.789600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.791177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94562-98df-4714-8369-406a8fd8a19f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.800694 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1329/2623] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:44.805125 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.815874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966b70-bc56-45f5-a089-f0f08e940d91 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:44.820492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:44.821398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.821688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.821989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] PATH_INFO: `/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440/users/cf47305a96664a67a2df9df23c08eff4/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.823958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a23cb1a0c6147e6b0a156cbc1c30440, user_id=cf47305a96664a67a2df9df23c08eff4, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:44.826645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:44.837613 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1295/2624] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:44 2022] PUT /identity/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440/users/cf47305a96664a67a2df9df23c08eff4/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:44.847663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b55fb-060d-4559-ba85-4910199d384a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:44.847956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b55fb-060d-4559-ba85-4910199d384a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:44.848238 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b55fb-060d-4559-ba85-4910199d384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:44.876177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-117b55fb-060d-4559-ba85-4910199d384a None None] MFA Rules not processed for user `cf47305a96664a67a2df9df23c08eff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:44.877418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-117b55fb-060d-4559-ba85-4910199d384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:44.898093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:44.949214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1330/2625] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:44.960842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:44.962658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:45.020347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f0d45-c255-41f8-b82f-a6db52f473cd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:45.024871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:45.025550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:45.025896 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:45.026150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:45.084452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:45.085974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:45.091210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:45.097207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1296/2626] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:46.624606 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b55fb-060d-4559-ba85-4910199d384a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:46.626035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-117b55fb-060d-4559-ba85-4910199d384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:46.638389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b55fb-060d-4559-ba85-4910199d384a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:46.641975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:46.642620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:46.642911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:46.643185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:46.699278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:46.700637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:46.710243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1331/2627] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:47.226783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:47.235963 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d329441-3249-4cdd-bd52-cef611f3c1f1 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:47.239525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:47.240697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:47.240928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:47.241200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:47.250750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:47.252495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7ca0-3aac-45cf-8b9f-bedb0e9a6e01 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:47.263807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1297/2628] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:51.103528 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:51.163859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a6bfc-919f-4a76-a976-59599d76fe13 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:51.168308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:51.169166 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:51.169480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:51.169762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] PATH_INFO: `/v3/users/ffbee363db704dc3a6e68961bf62260a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:51.170956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbee363db704dc3a6e68961bf62260a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:51.172891 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:51.218347 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbee363db704dc3a6e68961bf62260a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.227067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbee363db704dc3a6e68961bf62260a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.232357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:51.243080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbee363db704dc3a6e68961bf62260a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.243943 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbee363db704dc3a6e68961bf62260a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.251967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbee363db704dc3a6e68961bf62260a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.259401 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1332/2629] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:51 2022] DELETE /identity/v3/users/ffbee363db704dc3a6e68961bf62260a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:51.264097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:51.264588 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:51.264684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:51.283707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:51.284921 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:51.348225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1298/2630] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:51.359660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:51.361408 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:51.425026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdde22-7de7-40df-b09b-417a0899da3e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:51.429134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:51.429821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:51.430108 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:51.430378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:51.439356 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:51.440770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:51.450584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1333/2631] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:51.894303 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:51.895779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:51.908305 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9c7f8-2374-4f93-8d08-b37e55d1ad81 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:51.913972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:51.914859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:51.915169 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:51.915385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] PATH_INFO: `/v3/projects/b42097a844764c388a30caf0bd86cb4b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:51.917275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42097a844764c388a30caf0bd86cb4b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:51.919447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:51.970968 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42097a844764c388a30caf0bd86cb4b', 'request_id': 'req-d299928c-fed9-4e86-af57-9ad64b8995c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.981023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42097a844764c388a30caf0bd86cb4b', 'request_id': 'req-d299928c-fed9-4e86-af57-9ad64b8995c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.986587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42097a844764c388a30caf0bd86cb4b', 'request_id': 'req-d299928c-fed9-4e86-af57-9ad64b8995c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.998148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] The token cache is being invalidate because project b42097a844764c388a30caf0bd86cb4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:51.998440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:51.999102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.000645 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1299/2632] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:51 2022] DELETE /identity/v3/projects/b42097a844764c388a30caf0bd86cb4b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:52.004579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:52.015379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b913ea-890e-4ee8-9639-671c29a4b9be service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:52.086537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:52.087516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:52.087738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:52.088016 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] PATH_INFO: `/v3/users/fc8bab25a65240839f10b739b52154dc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:52.089272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8bab25a65240839f10b739b52154dc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:52.091256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:52.146908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8bab25a65240839f10b739b52154dc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.159222 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8bab25a65240839f10b739b52154dc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.189833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8bab25a65240839f10b739b52154dc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.190567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8bab25a65240839f10b739b52154dc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.200797 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8bab25a65240839f10b739b52154dc'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.202695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:52.220964 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1334/2633] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:52 2022] DELETE /identity/v3/users/fc8bab25a65240839f10b739b52154dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:52.532905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:52.534383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:52.547805 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299928c-fed9-4e86-af57-9ad64b8995c7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:52.606446 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:52.607318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:52.607603 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:52.607880 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] PATH_INFO: `/v3/projects/178915e320884ce79663c8f0466d1560` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:52.609077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=178915e320884ce79663c8f0466d1560)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:52.611049 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:52.622922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:52.657746 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178915e320884ce79663c8f0466d1560', 'request_id': 'req-f3030a48-3f8e-4f24-aa9e-736920259c5b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.665640 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178915e320884ce79663c8f0466d1560', 'request_id': 'req-f3030a48-3f8e-4f24-aa9e-736920259c5b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.671700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178915e320884ce79663c8f0466d1560', 'request_id': 'req-f3030a48-3f8e-4f24-aa9e-736920259c5b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.682763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] The token cache is being invalidate because project 178915e320884ce79663c8f0466d1560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:52.683054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.683712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.685227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1300/2634] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:52 2022] DELETE /identity/v3/projects/178915e320884ce79663c8f0466d1560 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:52.689398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:52.690783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:52.703870 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100c07-a6a7-4cc8-b8a8-36e9ea1e51d7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:52.758331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:52.759220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:52.759510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:52.759793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] PATH_INFO: `/v3/users/cf47305a96664a67a2df9df23c08eff4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:52.760986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf47305a96664a67a2df9df23c08eff4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:52.762940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:52.808527 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf47305a96664a67a2df9df23c08eff4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.816055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf47305a96664a67a2df9df23c08eff4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.828248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf47305a96664a67a2df9df23c08eff4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.828876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf47305a96664a67a2df9df23c08eff4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.835739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c7134580-3ce1-41c5-acd6-183c04c46145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf47305a96664a67a2df9df23c08eff4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:52.841673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1335/2635] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:39:52 2022] DELETE /identity/v3/users/cf47305a96664a67a2df9df23c08eff4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:53.123684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.125158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.138779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030a48-3f8e-4f24-aa9e-736920259c5b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.203290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.204059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.204345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.204619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] PATH_INFO: `/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.205882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a23cb1a0c6147e6b0a156cbc1c30440)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.207762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.254009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a23cb1a0c6147e6b0a156cbc1c30440', 'request_id': 'req-6ec1a907-2041-4876-9798-3d282c01b4b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.262176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a23cb1a0c6147e6b0a156cbc1c30440', 'request_id': 'req-6ec1a907-2041-4876-9798-3d282c01b4b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.267871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a23cb1a0c6147e6b0a156cbc1c30440', 'request_id': 'req-6ec1a907-2041-4876-9798-3d282c01b4b4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.279348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] The token cache is being invalidate because project 5a23cb1a0c6147e6b0a156cbc1c30440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:39:53.279637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.280292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6ec1a907-2041-4876-9798-3d282c01b4b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:39:53.281853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1301/2636] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:39:53 2022] DELETE /identity/v3/projects/5a23cb1a0c6147e6b0a156cbc1c30440 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:53.287420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.287882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.287978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.306118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:53.307391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.368987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1336/2637] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:53.377762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.378050 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.378329 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.397250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:53.398544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.409043 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1302/2638] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:53.410719 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.426598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d27cf-ab45-4d4f-b4ae-932f272d6150 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.431578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.432258 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.432547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.432827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.433601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:39:53.433911 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.435514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.448115 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1337/2639] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:39:53.452482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.462758 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0af873-5c56-471f-85af-ed3c3d4e5862 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.467473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.468118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.468404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.468680 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.469420 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.471322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.496561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1303/2640] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:53.500824 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.507670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:53.511541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfee11-7367-4c37-a62f-f919edfe1a39 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.577274 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.578194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.578560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.578898 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.579692 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.581944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.610495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1338/2641] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:53.611671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.617397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:53.624699 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f81e5-6337-49bf-aa11-1905acdf8053 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.638675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.648481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.648481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.648481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.648481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.648481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.654949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1304/2642] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:53.659391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.670403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccc10b-ca1a-49f6-8eb9-2ed0ae4511f7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.675014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.675679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.675969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.676249 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.676977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.679228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654427d8-3025-4279-865f-f1b48e2fbc9f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.690646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1339/2643] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:53.694887 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.705161 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1befc-9e4f-4ca8-ac10-65f7b847f72f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.709935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.710777 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.711059 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.711388 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] PATH_INFO: `/v3/projects/c49046dbb7514551999aed7e8aa2c3f6/users/9189d3ffa31549a8855ee10bbe77e458/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.726177 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49046dbb7514551999aed7e8aa2c3f6, user_id=9189d3ffa31549a8855ee10bbe77e458, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.728725 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.742544 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1305/2644] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:53 2022] PUT /identity/v3/projects/c49046dbb7514551999aed7e8aa2c3f6/users/9189d3ffa31549a8855ee10bbe77e458/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:39:53.753101 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.753417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.753704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.780375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] MFA Rules not processed for user `9189d3ffa31549a8855ee10bbe77e458`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:53.781619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:53.846735 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1340/2645] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:53.851230 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.916642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e73097-14e8-4759-9b57-bf218fac3897 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:53.924637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:53.925319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:53.925637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:53.925924 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:53.926829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:53.929263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:53.967547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1306/2646] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:39:53.971336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:53.985463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0999d-2c47-4d11-a019-1a4d44ccd567 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.047680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.048355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.048761 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.049074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.049790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.051725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.077036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1341/2647] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:53 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:54.078494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.089426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbee0d4-85df-42fe-a911-6978cdf65091 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.094123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.094789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.095076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.095354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.096072 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.097689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.108906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1307/2648] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:54.112861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.118703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:54.124676 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428a400-54ea-4670-ad10-ec90cb80707e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.129840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.131157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.131540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.131938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.132766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.134509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed9ad-76aa-4941-b221-7e8b1912ad4c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.148550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1342/2649] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:54.150097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.160753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3b1f-7609-4590-b992-8eca1a89c3ec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.168536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.169459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.169745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.170111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] PATH_INFO: `/v3/projects/223ba09d96ce41eb8edd4fc76033a88e/users/5e0ff1c606ca4de8a32f18664f345123/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.185795 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] RBAC: Authorizing `identity:create_grant(project_id=223ba09d96ce41eb8edd4fc76033a88e, user_id=5e0ff1c606ca4de8a32f18664f345123, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.188714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.202774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1308/2650] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:54 2022] PUT /identity/v3/projects/223ba09d96ce41eb8edd4fc76033a88e/users/5e0ff1c606ca4de8a32f18664f345123/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:54.214573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.214867 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.215144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.244148 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] MFA Rules not processed for user `5e0ff1c606ca4de8a32f18664f345123`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:54.245310 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:54.321932 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1343/2651] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:54.326679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.399529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd38475-0107-46d5-ade9-8df610c9048d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.405979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.406810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.407257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.407616 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.408576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.411399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.443158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1309/2652] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:39:54 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:54.447444 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.459355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157992-e232-43ab-9b36-4875b75ce2a1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.518865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.519361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.519647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.519929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.520629 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.523138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.548579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1344/2653] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:39:54 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:39:54.549872 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.555877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:54.562853 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c550cd-f4bc-4ef5-a6ca-3c8b34af723b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.569581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.570250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.570569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.570882 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.571697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.573447 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.587970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1310/2654] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:54.589231 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.600264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e95392-b23e-432b-a048-1bdd4ea2bb34 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.605509 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.606412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.606720 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.606999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] PATH_INFO: `/v3/projects/11c37961b50c4d65a1f378a4026f360c/users/590e03816aa64f529627223d37efc057/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.623168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c37961b50c4d65a1f378a4026f360c, user_id=590e03816aa64f529627223d37efc057, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.625878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765999e-9ea1-4025-b719-4cda7e676f65 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.641133 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1345/2655] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:54 2022] PUT /identity/v3/projects/11c37961b50c4d65a1f378a4026f360c/users/590e03816aa64f529627223d37efc057/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:54.645306 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.657618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411efe9-9dc7-41f6-acee-3a4c29576a76 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.698006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.698292 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.698618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.720620 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] MFA Rules not processed for user `294ef0f0e5e04c4193eba83a679afed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:54.722695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:54.733633 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.734846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.735571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.736140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.737351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.739128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.754729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1311/2656] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:39:54.760771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:54.773321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249546-8452-4cc7-80cb-072c5bb60a22 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:54.778897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:54.780346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.780846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.781323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] PATH_INFO: `/v3/projects/11c37961b50c4d65a1f378a4026f360c/users/590e03816aa64f529627223d37efc057/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.787342 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c37961b50c4d65a1f378a4026f360c, user_id=590e03816aa64f529627223d37efc057, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:54.792430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4617064-5fbb-4007-906b-42eb97b948ab admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:54.801970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:54.821017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1312/2657] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:39:54 2022] PUT /identity/v3/projects/11c37961b50c4d65a1f378a4026f360c/users/590e03816aa64f529627223d37efc057/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:39:54.822355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1346/2658] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:54.903735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:54.904032 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:54.904314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:54.960079 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] MFA Rules not processed for user `590e03816aa64f529627223d37efc057`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:39:54.961697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.080771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1313/2659] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:39:55.094975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:55.096536 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:55.169500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb1534-4a6b-4e05-97b2-02e6bde7e866 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:55.234378 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:55.234734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:55.235083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:55.235441 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:55.305745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:55.307362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:55.318643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1347/2660] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:56.965098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:56.966864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:56.982351 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6770e9-5ecf-440b-894b-d587035671a5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:56.986442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:56.987091 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:56.987375 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:56.987828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.059852 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.061200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.065320 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:39:57.072856 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1314/2661] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:57.640208 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:57.649581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de3a7-beda-4f98-b907-613f4e8408f6 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:57.653315 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:57.653999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:57.654345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:57.654632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:57.664345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:57.665834 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:57.676234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1348/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:59.370238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.371712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.437897 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae8a2e-d6b7-432a-ab89-e9a93d82d808 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:39:59.441859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:39:59.442536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:39:59.442821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:39:59.443100 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:39:59.444909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:39:59.514200 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:39:59.515696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:39:59.526263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1315/2663] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:39:59.986618 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:39:59.988699 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:00.003683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21675087-6bee-46de-a72f-7eca5f93c4aa service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:00.008389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:00.009050 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:00.009351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:00.009657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:00.011507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:00.063860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:00.087251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:00.088842 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff237a-29c4-4b9b-95e5-4498c407e4ef service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:00.105253 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1349/2664] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:01.684704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:01.717964 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea42e89-4d50-45a8-a9d9-5166df76051f service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:01.729396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:01.731256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.732490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.733681 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] PATH_INFO: `/v3/users/9189d3ffa31549a8855ee10bbe77e458` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.735462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9189d3ffa31549a8855ee10bbe77e458)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:01.739077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:01.831578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9189d3ffa31549a8855ee10bbe77e458'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:01.860246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9189d3ffa31549a8855ee10bbe77e458'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:01.861837 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9189d3ffa31549a8855ee10bbe77e458'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:01.882854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9189d3ffa31549a8855ee10bbe77e458'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:01.895124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9189d3ffa31549a8855ee10bbe77e458'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:01.905660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1316/2665] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:01 2022] DELETE /identity/v3/users/9189d3ffa31549a8855ee10bbe77e458 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:01.907702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:01.908920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:01.909219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:01.928076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:01.929305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:02.007828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1350/2666] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:02.018735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:02.019933 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:02.052824 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:02.116761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c05581-aeac-46f9-8504-2fcb4cef452b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:02.122919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:02.123989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:02.124275 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:02.124697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:02.138010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:02.139849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:02.155471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1317/2667] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:02.775968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:02.777446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:02.792678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcf4a0-b225-4226-9b6e-eed8e23a109f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:02.797550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:02.798677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:02.799139 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:02.799590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] PATH_INFO: `/v3/projects/c49046dbb7514551999aed7e8aa2c3f6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:02.801027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49046dbb7514551999aed7e8aa2c3f6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:02.803639 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:02.940648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49046dbb7514551999aed7e8aa2c3f6', 'request_id': 'req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:02.960599 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49046dbb7514551999aed7e8aa2c3f6', 'request_id': 'req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:02.972678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49046dbb7514551999aed7e8aa2c3f6', 'request_id': 'req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.020504 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] The token cache is being invalidate because project c49046dbb7514551999aed7e8aa2c3f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:03.021741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.025354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.030855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1351/2668] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:02 2022] DELETE /identity/v3/projects/c49046dbb7514551999aed7e8aa2c3f6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:03.031457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:03.052803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d608ce-bb09-43fb-8d7d-fc1511250d10 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:03.145393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:03.147092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:03.147539 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:03.147989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] PATH_INFO: `/v3/users/5e0ff1c606ca4de8a32f18664f345123` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:03.149590 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0ff1c606ca4de8a32f18664f345123)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:03.152525 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:03.226953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0ff1c606ca4de8a32f18664f345123'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.236634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0ff1c606ca4de8a32f18664f345123'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.237296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0ff1c606ca4de8a32f18664f345123'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.253153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0ff1c606ca4de8a32f18664f345123'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.263461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0ff1c606ca4de8a32f18664f345123'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:03.271754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1318/2669] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:03 2022] DELETE /identity/v3/users/5e0ff1c606ca4de8a32f18664f345123 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:03.988470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:03.990137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.008756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd65f5-06ae-45e0-a5df-4a89052a57d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:04.050977 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:04.052542 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.097185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:04.098555 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.099089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.099666 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] PATH_INFO: `/v3/users/3753498f22d5443ab0c01473c91a263c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.101422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3753498f22d5443ab0c01473c91a263c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:04.103986 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:04.120477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:04.123310 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:04.148641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9b865-5104-41d7-9b28-64ef37b26f1e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:04.154455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:04.155550 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.155989 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.156444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] PATH_INFO: `/v3/projects/223ba09d96ce41eb8edd4fc76033a88e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.158467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=223ba09d96ce41eb8edd4fc76033a88e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:04.160617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:04.176076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3753498f22d5443ab0c01473c91a263c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.188786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3753498f22d5443ab0c01473c91a263c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.221128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3753498f22d5443ab0c01473c91a263c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.222424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3753498f22d5443ab0c01473c91a263c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.231747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223ba09d96ce41eb8edd4fc76033a88e', 'request_id': 'req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.232779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3753498f22d5443ab0c01473c91a263c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.242318 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223ba09d96ce41eb8edd4fc76033a88e', 'request_id': 'req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.244736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1352/2670] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:03 2022] DELETE /identity/v3/users/3753498f22d5443ab0c01473c91a263c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:04.262012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223ba09d96ce41eb8edd4fc76033a88e', 'request_id': 'req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.294905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] The token cache is being invalidate because project 223ba09d96ce41eb8edd4fc76033a88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:04.295608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.298495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.300466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1319/2671] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:04 2022] DELETE /identity/v3/projects/223ba09d96ce41eb8edd4fc76033a88e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:04.305271 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:04.307092 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.448744 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023dfe-af44-4772-b0e3-f0ac9101405b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:04.455021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:04.456736 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.457141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.457612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] PATH_INFO: `/v3/users/590e03816aa64f529627223d37efc057` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.459280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=590e03816aa64f529627223d37efc057)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:04.461485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:04.516522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590e03816aa64f529627223d37efc057'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.526003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590e03816aa64f529627223d37efc057'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.539825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590e03816aa64f529627223d37efc057'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.540895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590e03816aa64f529627223d37efc057'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.549779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ace3dfe6-24fd-488c-b229-63bea7329fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590e03816aa64f529627223d37efc057'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.557827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1353/2672] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:04 2022] DELETE /identity/v3/users/590e03816aa64f529627223d37efc057 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:04.574126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:04.575783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.660846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ae680-73d6-4c0b-bdc7-6ee677a534ce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:04.722502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:04.723368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.723649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.724044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] PATH_INFO: `/v3/projects/7122215625b64963ab03b66f4c9d8a7f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.725150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7122215625b64963ab03b66f4c9d8a7f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:04.727099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:04.760393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:04.800456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7122215625b64963ab03b66f4c9d8a7f', 'request_id': 'req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.810494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7122215625b64963ab03b66f4c9d8a7f', 'request_id': 'req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.817508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7122215625b64963ab03b66f4c9d8a7f', 'request_id': 'req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.830201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] The token cache is being invalidate because project 7122215625b64963ab03b66f4c9d8a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:04.830481 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.831194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:04.836347 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1320/2673] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:04 2022] DELETE /identity/v3/projects/7122215625b64963ab03b66f4c9d8a7f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:04.838076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf20fe-3b9b-4f26-8ec7-982d4ab76b68 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.838373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf20fe-3b9b-4f26-8ec7-982d4ab76b68 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.838651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf20fe-3b9b-4f26-8ec7-982d4ab76b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.860672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f0cf20fe-3b9b-4f26-8ec7-982d4ab76b68 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:04.862373 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f0cf20fe-3b9b-4f26-8ec7-982d4ab76b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.871243 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:04.894450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:04.896467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.957119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1354/2674] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:04.965025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.965360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.965634 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.983951 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:04.985453 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:04.987477 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea7d71-2fa5-45bb-9b83-7d1b78c0e372 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:04.992791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:04.993713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:04.994004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:04.994282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] PATH_INFO: `/v3/projects/11c37961b50c4d65a1f378a4026f360c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:04.995494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c37961b50c4d65a1f378a4026f360c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:04.997085 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1355/2675] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:04.997581 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:04.998566 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.008539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e721-f29d-4f57-89dd-890273b7b046 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.013229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.015816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.015816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.015816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.016226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:05.016541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.018169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.030853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1356/2676] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:05.035180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.044921 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c221d-cfa4-43b8-81ed-11f6bdc7273f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.049736 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c37961b50c4d65a1f378a4026f360c', 'request_id': 'req-52055493-2af7-4afc-96e7-dac70c3d3500'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.069670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c37961b50c4d65a1f378a4026f360c', 'request_id': 'req-52055493-2af7-4afc-96e7-dac70c3d3500'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.076735 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c37961b50c4d65a1f378a4026f360c', 'request_id': 'req-52055493-2af7-4afc-96e7-dac70c3d3500'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.090679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] The token cache is being invalidate because project 11c37961b50c4d65a1f378a4026f360c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:05.090878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.091593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52055493-2af7-4afc-96e7-dac70c3d3500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:05.093313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1321/2677] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:04 2022] DELETE /identity/v3/projects/11c37961b50c4d65a1f378a4026f360c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:05.098265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98594c-2af0-4f76-b6bf-0295f80b82a8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.099825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98594c-2af0-4f76-b6bf-0295f80b82a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.099825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98594c-2af0-4f76-b6bf-0295f80b82a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.123217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3b98594c-2af0-4f76-b6bf-0295f80b82a8 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:05.124689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3b98594c-2af0-4f76-b6bf-0295f80b82a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.131094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.132615 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.132945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.133232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.134015 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.134976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1322/2678] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:05.141879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.143614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.143997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.144482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.163942 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:05.165185 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.174080 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1323/2679] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:05.178752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1357/2680] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:05.178915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.182937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.184907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.189671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ecfb4-f24c-43fb-b64c-0e0ae3fadf8a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.216052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:05.250804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.251561 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.251919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.252265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.253171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:05.253610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.255812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.257254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6c783-2b4f-4d19-9971-cb2d8f60b7a3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.262646 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.263493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.263802 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.264087 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.264817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.266803 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.271945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1324/2681] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:40:05.272778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.283385 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e42a33-c1ad-4737-9b53-221fa1c25ff0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.288812 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.289518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.289813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.290097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.290450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1358/2682] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:05.290796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.292807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.294357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.300957 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:05.305466 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30eab7f-24bb-441d-b61b-6f40fb39dc97 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.312312 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.313076 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.313369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.313896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.314655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.316247 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.321093 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1325/2683] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:05.325035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.326608 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1359/2684] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:05.330150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.340353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77d27d-fc96-41a9-9d21-c4e400a05bb8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.396839 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157706f7-02d3-47a8-a1e9-67868440fa3f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.405440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.405440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.405440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.405440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.405928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.407517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.409496 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.410375 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.410780 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.411161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.413370 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.417194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.429346 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1360/2685] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:05.436184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1326/2686] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:05.436450 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.437003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.446559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebced6c7-66f4-42e7-9dfb-5cab7f41e472 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.448090 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3ffa-036b-4b7b-954d-1bc5084c6292 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.451806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.452917 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.453052 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.453427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.453617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.453893 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] PATH_INFO: `/v3/projects/0d418ed7f92744978dd6a9f1f77287c5/users/64206d561ee4456a8f8c670dfd2944a5/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.454007 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.454172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.454966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.456498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.466939 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1327/2687] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:40:05.470834 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.471045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d418ed7f92744978dd6a9f1f77287c5, user_id=64206d561ee4456a8f8c670dfd2944a5, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.473756 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46036c6-f403-46f0-959a-61036ba8a37f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.482062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0432b2-db2c-4881-bf65-7fc69ae22790 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.490046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1361/2688] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:05 2022] PUT /identity/v3/projects/0d418ed7f92744978dd6a9f1f77287c5/users/64206d561ee4456a8f8c670dfd2944a5/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:05.494970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.495336 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.495680 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.512832 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:05.514077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.540840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.541541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.541829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.542113 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.542829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.545670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.561127 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1328/2689] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:05.566485 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.580492 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c73d-427f-4975-af4f-0244aee2dd48 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.587339 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.588256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.588570 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.588788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] PATH_INFO: `/v3/projects/5aff3f50b8894eea9f5ad306c93b239e/users/85f13b92a8aa46e3b06f7f8d31d5cbba/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.605173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1362/2690] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:05.605868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:05.616663 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.619056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.620194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aff3f50b8894eea9f5ad306c93b239e, user_id=85f13b92a8aa46e3b06f7f8d31d5cbba, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.623216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37e4fd-ce78-40f7-84f4-822e4070217f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.643030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1329/2691] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:05 2022] PUT /identity/v3/projects/5aff3f50b8894eea9f5ad306c93b239e/users/85f13b92a8aa46e3b06f7f8d31d5cbba/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:05.655614 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.655970 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.656628 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.702004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] MFA Rules not processed for user `85f13b92a8aa46e3b06f7f8d31d5cbba`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:05.702583 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f40de-b69b-468f-834c-2a9986b913bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.703499 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.725211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:05.762219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.762882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.763163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.763446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.787243 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1330/2692] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:05.805270 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:05.807258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:05.829641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.831059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.842046 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1363/2693] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:05.897316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49942cc-0181-4ab0-96f9-81b5a225fd9d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:05.901240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:05.903022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:05.903022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:05.903022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:05.914437 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:05.915876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c517ae-8c04-4b7e-afb7-b7e0415bf190 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:05.926258 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1331/2694] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:40:06.871810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:06.873521 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:06.894748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6650-2044-4bcd-8d55-00a4832501e0 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:06.898970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:06.899991 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:06.900490 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:06.900970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:06.908491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:06.931275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:06.933477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:06.946119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1364/2695] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:11.343015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.343530 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.343639 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.391756 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] MFA Rules not processed for user `64206d561ee4456a8f8c670dfd2944a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:11.393042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.569528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1332/2696] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:11.581168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.597641 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:11.599765 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.614585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31a88-4396-4234-a0bf-f99c971d2418 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.624051 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.624638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.624989 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.625353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.626369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:11.629067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:11.774134 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.775484 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24512bcd-23aa-4c2c-8c35-ee6796d843d9 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:11.801017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a891858-27e6-46a7-92ea-ae200351b859 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:11.807132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1365/2697] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:11.812189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:11.812858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:11.813138 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:11.813730 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:11.841150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:11.841150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:11.867099 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1333/2698] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:13.961610 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:13.962114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:13.962219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:13.986443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] MFA Rules not processed for user `64206d561ee4456a8f8c670dfd2944a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:13.988939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:13.999799 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1366/2699] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:16.793521 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:16.795081 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:16.815285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515de7c-a9f1-49f8-bc91-4e2c97a222d1 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:16.822077 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:16.823353 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:16.823789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:16.824271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] PATH_INFO: `/v3/users/7d098bacbe444141b25a14d9d3d06cc1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:16.825724 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d098bacbe444141b25a14d9d3d06cc1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:16.827835 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:16.889578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d098bacbe444141b25a14d9d3d06cc1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:16.899229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d098bacbe444141b25a14d9d3d06cc1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:16.902408 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d098bacbe444141b25a14d9d3d06cc1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:16.928234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d098bacbe444141b25a14d9d3d06cc1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:16.936800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d098bacbe444141b25a14d9d3d06cc1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:16.945372 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1334/2700] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:16 2022] DELETE /identity/v3/users/7d098bacbe444141b25a14d9d3d06cc1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.174937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.191906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fc975-c013-4e7f-9380-473e03f238fa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.251747 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.264131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.265165 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.265485 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.265764 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] PATH_INFO: `/v3/projects/c82ac9fcca384f0793c21f7e06b234bc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.267114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82ac9fcca384f0793c21f7e06b234bc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.269026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.322250 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82ac9fcca384f0793c21f7e06b234bc', 'request_id': 'req-ab229c94-1316-4250-a26d-66ad7ca19460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.330057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82ac9fcca384f0793c21f7e06b234bc', 'request_id': 'req-ab229c94-1316-4250-a26d-66ad7ca19460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.336709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82ac9fcca384f0793c21f7e06b234bc', 'request_id': 'req-ab229c94-1316-4250-a26d-66ad7ca19460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.349146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] The token cache is being invalidate because project c82ac9fcca384f0793c21f7e06b234bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:17.349473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.350143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.353384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1367/2701] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:17 2022] DELETE /identity/v3/projects/c82ac9fcca384f0793c21f7e06b234bc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.356054 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.357553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.371108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5677-ea4d-4cc1-a0ac-6bb98702d62d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.432189 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.433076 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.433395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.433682 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] PATH_INFO: `/v3/users/c51cdc6c95bc4bef9db8ea023de08ec8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.434932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51cdc6c95bc4bef9db8ea023de08ec8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.437256 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.448914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:17.485195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51cdc6c95bc4bef9db8ea023de08ec8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.493095 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51cdc6c95bc4bef9db8ea023de08ec8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.493833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51cdc6c95bc4bef9db8ea023de08ec8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.505905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51cdc6c95bc4bef9db8ea023de08ec8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.512845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ff4fd8a9-1608-4598-8553-82c6d403a52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51cdc6c95bc4bef9db8ea023de08ec8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.518770 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1335/2702] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:17 2022] DELETE /identity/v3/users/c51cdc6c95bc4bef9db8ea023de08ec8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:17.690433 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:17.692066 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:17.706104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229c94-1316-4250-a26d-66ad7ca19460 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:17.781141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:17.781959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:17.782244 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:17.782522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] PATH_INFO: `/v3/projects/4f591e6efff3443b9561899c037aa719` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:17.783691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f591e6efff3443b9561899c037aa719)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:17.785632 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:17.830551 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f591e6efff3443b9561899c037aa719', 'request_id': 'req-d04c224c-2579-479b-8f28-aad1fc9c45d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.837721 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f591e6efff3443b9561899c037aa719', 'request_id': 'req-d04c224c-2579-479b-8f28-aad1fc9c45d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.843293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f591e6efff3443b9561899c037aa719', 'request_id': 'req-d04c224c-2579-479b-8f28-aad1fc9c45d7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.854766 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] The token cache is being invalidate because project 4f591e6efff3443b9561899c037aa719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:17.855059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.855695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:17.857213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1368/2703] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:17 2022] DELETE /identity/v3/projects/4f591e6efff3443b9561899c037aa719 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:19.149655 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ecc7-a9ab-4331-8ee4-7d09c0850ea6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:19.150134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ecc7-a9ab-4331-8ee4-7d09c0850ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:19.150252 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ecc7-a9ab-4331-8ee4-7d09c0850ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:19.168564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e8c7ecc7-a9ab-4331-8ee4-7d09c0850ea6 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:19.169803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e8c7ecc7-a9ab-4331-8ee4-7d09c0850ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:19.227702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1336/2704] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:19.277201 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:19.280054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:19.295947 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:19.359330 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c224c-2579-479b-8f28-aad1fc9c45d7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:19.366813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:19.367792 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:19.368275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:19.368816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:19.378326 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:19.380285 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:19.390351 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1369/2705] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:40:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:40:21.330938 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a589a-300a-4939-92d4-ccf8d48dfee5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:21.331494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a589a-300a-4939-92d4-ccf8d48dfee5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:21.331618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a589a-300a-4939-92d4-ccf8d48dfee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:21.359958 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5a0a589a-300a-4939-92d4-ccf8d48dfee5 None None] MFA Rules not processed for user `42165c1e1b324fedbb926ac20f0f920a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:21.361383 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5a0a589a-300a-4939-92d4-ccf8d48dfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:21.445819 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1337/2706] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:22.116810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.119229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.320111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a993bc-d477-4b92-a294-41f76aa58ac8 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.327487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.329405 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.329969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.330426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] PATH_INFO: `/v3/users/85f13b92a8aa46e3b06f7f8d31d5cbba` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.331863 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f13b92a8aa46e3b06f7f8d31d5cbba)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.334011 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.420748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f13b92a8aa46e3b06f7f8d31d5cbba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.423751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:22.433361 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f13b92a8aa46e3b06f7f8d31d5cbba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.465936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f13b92a8aa46e3b06f7f8d31d5cbba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.466684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f13b92a8aa46e3b06f7f8d31d5cbba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.483794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f13b92a8aa46e3b06f7f8d31d5cbba'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:22.506808 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1370/2707] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:22 2022] DELETE /identity/v3/users/85f13b92a8aa46e3b06f7f8d31d5cbba => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:22.508142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.508619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.508788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.560935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:22.576774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:22.579004 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.695646 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1338/2708] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:22.702495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:22.704128 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:22.806058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc42af-bdd6-4917-a86c-e72fb29c4157 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:22.809774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:22.810494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:22.810850 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:22.811290 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:22.827308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:22.828873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aca83-f939-4aad-bc04-766af873522d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:22.840805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1371/2709] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:23.595034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:23.596533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:23.632373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133325-78b0-497c-a6f4-4508a5eeb982 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:23.640205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:23.641271 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.641598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.641881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] PATH_INFO: `/v3/projects/5aff3f50b8894eea9f5ad306c93b239e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.643078 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aff3f50b8894eea9f5ad306c93b239e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:23.644972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:23.699030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aff3f50b8894eea9f5ad306c93b239e', 'request_id': 'req-f04da699-e505-452c-899f-64037c2504e4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.708421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aff3f50b8894eea9f5ad306c93b239e', 'request_id': 'req-f04da699-e505-452c-899f-64037c2504e4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.718784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aff3f50b8894eea9f5ad306c93b239e', 'request_id': 'req-f04da699-e505-452c-899f-64037c2504e4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.731373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] The token cache is being invalidate because project 5aff3f50b8894eea9f5ad306c93b239e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:23.731704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.732369 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f04da699-e505-452c-899f-64037c2504e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:23.734030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1339/2710] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:23 2022] DELETE /identity/v3/projects/5aff3f50b8894eea9f5ad306c93b239e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:23.740032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.740531 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.740654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.852575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:23.853829 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:23.965435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1372/2711] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:23.970817 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:23.971109 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:23.971393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:23.997940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:23.999234 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:24.030002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1340/2712] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:24.034712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.058467 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026f23-8502-415e-9219-12e6c7437a15 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.077979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.078869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.079275 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.079614 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.080647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:24.081162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.083539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.129251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1373/2713] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:40:24.131186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.142660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008de57c-4bed-4b41-b4ab-971a46a3bf81 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.150752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.151469 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.151737 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.152016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.152717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.156013 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.224669 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1341/2714] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/projects => generated 391 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:24.226677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.252941 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab0e4-e5e5-442e-816c-9d28626c2ff6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.296518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:24.408871 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.410058 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.410511 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.411635 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.412497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.414627 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.445895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1374/2715] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/users => generated 296 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:24.449974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.464769 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39a126-ae5c-4de6-83c4-8bdeb4b9d43f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.469653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.470336 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.470845 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.471128 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.471857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.474123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.487910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1342/2716] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:24.492353 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.503317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c407f-41d1-408d-af99-60a12bb09954 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.511103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.512554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.512877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.513174 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.514371 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.521790 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0485e-c605-4826-ad07-2638a230d161 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.533318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1375/2717] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:24.537844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.548866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059fbe-5ca2-4a29-9bc3-3e22fcbe6bd6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:24.554527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:24.555672 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.556062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.556649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] PATH_INFO: `/v3/projects/a14e2f16f73b44929e41270d4c0f6dad/users/8378a96ecf0640fbac47b80587d5e3c6/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.574444 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14e2f16f73b44929e41270d4c0f6dad, user_id=8378a96ecf0640fbac47b80587d5e3c6, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:24.580443 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:24.604748 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1343/2718] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:24 2022] PUT /identity/v3/projects/a14e2f16f73b44929e41270d4c0f6dad/users/8378a96ecf0640fbac47b80587d5e3c6/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:40:24.624316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77b38-bc54-4f42-a5ee-0d623e568c47 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:24.627855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77b38-bc54-4f42-a5ee-0d623e568c47 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:24.649461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b77b38-bc54-4f42-a5ee-0d623e568c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:24.701257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-89b77b38-bc54-4f42-a5ee-0d623e568c47 None None] MFA Rules not processed for user `8378a96ecf0640fbac47b80587d5e3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:24.702077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-89b77b38-bc54-4f42-a5ee-0d623e568c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:24.927047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1376/2719] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:24.939556 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:24.941390 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:24.980348 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:25.049244 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490a21-5a95-4c60-bb35-1b5663a58ba5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:25.068803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:25.070788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:25.071498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:25.071911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:25.092704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:25.094389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:25.114813 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1344/2720] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:34.922451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:35.101217 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=15 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:39.859856 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:39.860324 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:39.860435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:39.891503 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:39.892811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:39.973494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1377/2721] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:40:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:39.982207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:39.983610 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:40.071374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317054c-ac17-47e9-a6c6-307bf6d8b2fc service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:40.079006 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:40.079665 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:40.079944 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:40.080223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:40.090419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:40.092066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:40.111731 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1345/2722] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:40:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:40:42.253164 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.255472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:42.328637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54134edf-cbf0-4a2e-9c13-f755d9325033 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.333983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.334936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.335216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.335494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] PATH_INFO: `/v3/users/294ef0f0e5e04c4193eba83a679afed1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.336765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] RBAC: Authorizing `identity:delete_user(user_id=294ef0f0e5e04c4193eba83a679afed1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.338763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.401029 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294ef0f0e5e04c4193eba83a679afed1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.411638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294ef0f0e5e04c4193eba83a679afed1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.429394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294ef0f0e5e04c4193eba83a679afed1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.430298 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294ef0f0e5e04c4193eba83a679afed1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.437770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294ef0f0e5e04c4193eba83a679afed1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.445004 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1378/2723] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:42 2022] DELETE /identity/v3/users/294ef0f0e5e04c4193eba83a679afed1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:42.621720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:42.633671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ef2ea-23f2-414a-8469-9fd485834071 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:42.696482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:42.697359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:42.697670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:42.697951 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] PATH_INFO: `/v3/projects/f8ff4496a9884d6ba5f53803d4aa475b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:42.699167 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ff4496a9884d6ba5f53803d4aa475b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:42.701061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:42.747092 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ff4496a9884d6ba5f53803d4aa475b', 'request_id': 'req-96f76e91-60a8-47d9-a739-dfd43a2cd58b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.755760 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ff4496a9884d6ba5f53803d4aa475b', 'request_id': 'req-96f76e91-60a8-47d9-a739-dfd43a2cd58b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.761679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ff4496a9884d6ba5f53803d4aa475b', 'request_id': 'req-96f76e91-60a8-47d9-a739-dfd43a2cd58b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.774155 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] The token cache is being invalidate because project f8ff4496a9884d6ba5f53803d4aa475b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:42.774431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.774979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:42.776509 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1346/2724] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:42 2022] DELETE /identity/v3/projects/f8ff4496a9884d6ba5f53803d4aa475b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:43.549032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:43.550700 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.563269 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:43.632430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab79aeb-21e1-4f34-acf4-5a6f157c516b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:43.680741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:43.681418 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:43.681704 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:43.681982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:43.683771 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:43.744662 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:43.746080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35565244-f2a4-4e55-b29a-e79425dd981a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:43.759923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1379/2725] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:45.025034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.026517 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.050476 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.097176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f76e91-60a8-47d9-a739-dfd43a2cd58b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.102687 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.103621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.103906 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.104188 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] PATH_INFO: `/v3/users/8378a96ecf0640fbac47b80587d5e3c6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.105467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8378a96ecf0640fbac47b80587d5e3c6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.107694 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.159262 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8378a96ecf0640fbac47b80587d5e3c6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.167265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8378a96ecf0640fbac47b80587d5e3c6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.168023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8378a96ecf0640fbac47b80587d5e3c6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.181142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8378a96ecf0640fbac47b80587d5e3c6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.188439 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8378a96ecf0640fbac47b80587d5e3c6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.196474 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1347/2726] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:40:45 2022] DELETE /identity/v3/users/8378a96ecf0640fbac47b80587d5e3c6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:45.200885 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.201435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.201540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.219855 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:45.221094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.271748 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.283912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1380/2727] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:45.293920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.295331 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.357901 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fdcc3-2c92-4d0b-90ed-8116cdf55873 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.361776 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.362428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.362712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.362990 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.372392 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.373931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b79cf-f8bf-45c9-8a68-d9c04fc98a74 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.384368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1348/2728] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:45.684948 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.687193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.706584 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8050a-a7d2-4499-8184-1d0b056e25c1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.712039 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.712905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.713191 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.713508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] PATH_INFO: `/v3/projects/a14e2f16f73b44929e41270d4c0f6dad` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.714709 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14e2f16f73b44929e41270d4c0f6dad)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.716638 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.771056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14e2f16f73b44929e41270d4c0f6dad', 'request_id': 'req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.781384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14e2f16f73b44929e41270d4c0f6dad', 'request_id': 'req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.791161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14e2f16f73b44929e41270d4c0f6dad', 'request_id': 'req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.803684 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] The token cache is being invalidate because project a14e2f16f73b44929e41270d4c0f6dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:40:45.803977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.804668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1368094d-4f0e-4c8b-ae27-88f1c5f8cd8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:40:45.810241 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1381/2729] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:40:45 2022] DELETE /identity/v3/projects/a14e2f16f73b44929e41270d4c0f6dad => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:45.812123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.812421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.812754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.832433 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:45.833750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.882379 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:45.909037 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1349/2730] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:45.917412 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.918119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.918468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.939768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:45.941070 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:45.950125 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1382/2731] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:45.954417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:45.964720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc1ee5-f320-4f39-a6cf-0a5fb6171e17 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:45.976557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:45.977454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:45.978140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:45.978345 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:45.979334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:40:45.979752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:45.982560 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:45.997391 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1350/2732] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:40:45.999903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.009607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e810-c544-4b39-ab61-84fde568fc9b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.015316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.015992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.016264 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.016542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.017318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.020163 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.050286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1383/2733] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:45 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:46.051430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.062533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f5dd-c333-4630-a639-dc47c344cad3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.127966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.128656 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.128943 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.129825 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.130527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.132503 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.161221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1351/2734] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:46 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:46.162783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.175182 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8777ee-f3d1-4c68-95f1-68370a07193a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.181195 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.182184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.182184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.182500 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.183393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.185075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.203813 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1384/2735] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:46.205894 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.219182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847425-c1c0-4a12-bc3d-47fd9402b359 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.225450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.226129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.226412 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.226826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.227617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.229458 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de5dfb-efc0-449b-9e12-0c061399caa0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.241945 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1352/2736] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:46.245967 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.257683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d692986-08f2-429a-96e4-55c5560479ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.263950 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.264266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.264644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.265045 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] PATH_INFO: `/v3/projects/51b448bb6d0542e9aeb90961739d9c06/users/2a821209feb04c0a9f67aa6dfe7fb1e9/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.287971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b448bb6d0542e9aeb90961739d9c06, user_id=2a821209feb04c0a9f67aa6dfe7fb1e9, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.291542 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.306539 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1385/2737] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:46 2022] PUT /identity/v3/projects/51b448bb6d0542e9aeb90961739d9c06/users/2a821209feb04c0a9f67aa6dfe7fb1e9/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:46.311692 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d564-6c4c-4b85-b898-5dc04c13a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.312063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d564-6c4c-4b85-b898-5dc04c13a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.312419 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25d564-6c4c-4b85-b898-5dc04c13a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.331418 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eb25d564-6c4c-4b85-b898-5dc04c13a2c3 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:46.332821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eb25d564-6c4c-4b85-b898-5dc04c13a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:46.396296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1353/2738] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:46.406041 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:46.407965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:46.469587 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912f619-eb95-4666-8e91-1f0d17edd86a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:46.473142 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:46.473841 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:46.474129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:46.474406 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:46.483953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:46.485425 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:46.496303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1386/2739] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:50.545582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.546057 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.546154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.575649 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] MFA Rules not processed for user `2a821209feb04c0a9f67aa6dfe7fb1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:50.577450 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:50.624368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:50.652798 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1354/2740] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:50.657350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.667868 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c6b2d-0f71-4b87-9d70-3f540a53c99a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.672619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.673422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.673712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.674019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.674714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.677214 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:50.692809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:50.712366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1387/2741] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:40:50 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:40:50.713565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.724689 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5b430-6a8d-44d7-af80-9d9197915c75 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.786876 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.787584 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.787869 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.788145 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.788910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.790903 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:50.815164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1355/2742] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:40:50 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:40:50.819410 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.832713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e66c7-c85b-42e5-ade7-b48e4e1b4478 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.837690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.838385 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.838667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.838942 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.839671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.841287 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:50.853391 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1388/2743] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:50.856953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.868148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25521b-7356-4a03-a80d-9783bbf54b31 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.873034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.873965 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.874257 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.874529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] PATH_INFO: `/v3/projects/04c862f2784c4a999e5c19c6826bdb6c/users/06bf933dbd03458ab93fb9f598cc522e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.890804 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c862f2784c4a999e5c19c6826bdb6c, user_id=06bf933dbd03458ab93fb9f598cc522e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.893322 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:50.909579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1356/2744] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:50 2022] PUT /identity/v3/projects/04c862f2784c4a999e5c19c6826bdb6c/users/06bf933dbd03458ab93fb9f598cc522e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:50.914540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:50.925118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079216-3398-4625-b98a-cf60a2aa71ce admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:50.987939 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:50.988653 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:50.988940 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:50.989216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:50.989988 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:50.991590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ac4db-9090-45d4-b2b3-9ee986249306 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:51.002682 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1389/2745] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:40:51.006850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:51.017750 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18daa4c8-4850-4d5b-a5f9-e0590ee75799 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:51.022973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:51.023999 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:51.024354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:51.024647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] PATH_INFO: `/v3/projects/04c862f2784c4a999e5c19c6826bdb6c/users/06bf933dbd03458ab93fb9f598cc522e/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:51.026718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c862f2784c4a999e5c19c6826bdb6c, user_id=06bf933dbd03458ab93fb9f598cc522e, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:51.029425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:51.041022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1357/2746] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:40:51 2022] PUT /identity/v3/projects/04c862f2784c4a999e5c19c6826bdb6c/users/06bf933dbd03458ab93fb9f598cc522e/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:40:55.395234 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646886d-46ac-4d08-b7fc-96dd73fa13c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:55.396103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646886d-46ac-4d08-b7fc-96dd73fa13c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:55.396389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646886d-46ac-4d08-b7fc-96dd73fa13c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:55.426389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1646886d-46ac-4d08-b7fc-96dd73fa13c5 None None] MFA Rules not processed for user `06bf933dbd03458ab93fb9f598cc522e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:55.427668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1646886d-46ac-4d08-b7fc-96dd73fa13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:55.501590 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1390/2747] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:40:55.512658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:55.514406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:55.580688 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22d640-fd74-41be-bed4-5df3d3703e8e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:55.584389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0df73889cb3f4bb58bdb5d674ed5379d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:55.585693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:55.585693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:55.585693 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:55.643163 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:40:55.667695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:55.669190 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:55.679074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1358/2748] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:40:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:40:55.774394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9552a-0c72-4061-a6c4-c06e854df0c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:55.774705 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9552a-0c72-4061-a6c4-c06e854df0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:55.774970 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9552a-0c72-4061-a6c4-c06e854df0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:55.792990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-67c9552a-0c72-4061-a6c4-c06e854df0c5 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:40:55.794232 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-67c9552a-0c72-4061-a6c4-c06e854df0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:55.849088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1391/2749] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:40:55.865304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:40:55.866744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:40:55.940245 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e7a2d-90a2-48ec-bc0f-9f4fc650b5c1 service glance] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:40:55.945654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:40:55.946307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:40:55.946593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:40:55.946871 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:40:55.955571 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:40:55.957619 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2ec87-98fa-4fd3-b7a0-3a9331cf5be1 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:40:55.967811 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1359/2750] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:41:04.651263 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:04.651770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:04.651902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:04.668959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] MFA Rules not processed for user `2a821209feb04c0a9f67aa6dfe7fb1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:04.670175 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:04.677386 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1392/2751] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:05.964600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=39 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:06.598229 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.598513 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.598791 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.617647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:41:06.618873 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:06.626378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1360/2752] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Jul 12 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:41:06.639494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:06.648578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:06.653116 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71b1df-5304-48f1-9726-64d32e67a11c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:06.658554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:06.659216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:06.659498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:06.659775 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:06.669640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:06.671230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:06.679983 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1393/2753] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:41:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 14:41:16.611846 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:16.676522 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:27.954729 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:27.956215 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:28.051563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9b71-f983-40d8-be17-6178de1c9662 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:28.056885 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:28.058205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:28.058494 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:28.058772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:28.060562 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:41:28.073197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:28.074541 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:28.085402 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1361/2754] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:35.269257 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:41:35.324168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e064b-edb0-4e63-aee8-acb73fe038f7 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:41:35.327741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:41:35.328389 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:41:35.328674 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:41:35.328956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:41:35.338167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:41:35.339660 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:41:35.349667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1394/2755] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:41:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:41:38.081820 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=41 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:41:45.344887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=26 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:27.450583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:27.452069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:27.463910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f33d07-f5ea-41fb-b4ae-7c0140af9490 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:27.467488 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:27.468119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:27.468403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:27.468675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:27.471000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:27.551941 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:27.553404 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:27.573916 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1362/2756] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:32.315846 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:32.324888 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026159cc-6a96-4268-a167-6f9d10dccd20 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:32.329229 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:32.330739 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:32.331439 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:32.331922 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:32.334078 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:32.355193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:32.356809 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:32.371282 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1395/2757] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:42:35.354912 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:35.365226 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c6ca-8d35-414b-a593-e1a46df02e41 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:35.369201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:35.369954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:35.370307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:35.370676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:35.372675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:35.447759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:35.449423 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:35.462827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1363/2758] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:41.778981 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:41.780737 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:41.799772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9fe18-2e4e-4314-98a6-ee1c548d342f service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:41.804495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:41.805515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:41.805810 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:41.806089 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] PATH_INFO: `/v3/users/42165c1e1b324fedbb926ac20f0f920a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:41.807289 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42165c1e1b324fedbb926ac20f0f920a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:41.809237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:41.856160 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42165c1e1b324fedbb926ac20f0f920a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:41.863654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42165c1e1b324fedbb926ac20f0f920a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:41.877349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42165c1e1b324fedbb926ac20f0f920a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:41.878027 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42165c1e1b324fedbb926ac20f0f920a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:41.884793 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42165c1e1b324fedbb926ac20f0f920a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:41.890691 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1396/2759] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:41 2022] DELETE /identity/v3/users/42165c1e1b324fedbb926ac20f0f920a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:42.065449 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:42.075982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b783-73b5-4fd2-839e-cfb97870e43e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:42.137166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:42.137980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:42.138260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:42.138536 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] PATH_INFO: `/v3/projects/aac705367c454623a23f11bfa8d25756` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:42.139714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac705367c454623a23f11bfa8d25756)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:42.141790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:42.143288 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:42.192102 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac705367c454623a23f11bfa8d25756', 'request_id': 'req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.199440 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac705367c454623a23f11bfa8d25756', 'request_id': 'req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.205031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac705367c454623a23f11bfa8d25756', 'request_id': 'req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.216114 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] The token cache is being invalidate because project aac705367c454623a23f11bfa8d25756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:42.216401 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.217035 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.218576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1364/2760] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:42 2022] DELETE /identity/v3/projects/aac705367c454623a23f11bfa8d25756 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:42.223119 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:42.224538 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:42.238358 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4dc70-afbb-46c2-b46c-7b99e73eb3e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:42.297026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:42.298570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:42.298925 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:42.299340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] PATH_INFO: `/v3/users/16d7162fcec44e00a4078735725b124b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:42.300657 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d7162fcec44e00a4078735725b124b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:42.302636 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:42.356578 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d7162fcec44e00a4078735725b124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.365114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d7162fcec44e00a4078735725b124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.370782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:42.379369 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d7162fcec44e00a4078735725b124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.380144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d7162fcec44e00a4078735725b124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.387398 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d7162fcec44e00a4078735725b124b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.393421 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1397/2761] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:42:42 2022] DELETE /identity/v3/users/16d7162fcec44e00a4078735725b124b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:42.575822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:42.577484 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:42.595826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0849e59-1d50-4692-8ea6-ccfd6d3e0331 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:42.656743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:42.657583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:42.657862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:42.658136 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] PATH_INFO: `/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:42.659319 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a90fcd1e654fb09b6d48c1f6c5d6f2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:42.661359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:42.712874 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a90fcd1e654fb09b6d48c1f6c5d6f2', 'request_id': 'req-d96fac16-8178-4d9e-abd8-633cab2194c9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.720862 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a90fcd1e654fb09b6d48c1f6c5d6f2', 'request_id': 'req-d96fac16-8178-4d9e-abd8-633cab2194c9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.726405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a90fcd1e654fb09b6d48c1f6c5d6f2', 'request_id': 'req-d96fac16-8178-4d9e-abd8-633cab2194c9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.737919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] The token cache is being invalidate because project a1a90fcd1e654fb09b6d48c1f6c5d6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:42:42.738207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.738833 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:42:42.740428 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1365/2762] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:42:42 2022] DELETE /identity/v3/projects/a1a90fcd1e654fb09b6d48c1f6c5d6f2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:42:49.379553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:49.381121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:49.464387 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0e55e-34b7-4ab7-b4dc-095d051ea02a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:49.523384 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:49.524061 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:49.524381 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:49.524734 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:49.527479 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:49.538278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:49.539843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f70a9e-8615-405a-b037-826aa638d4de service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:49.549887 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1398/2763] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:52.734396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:42:55.914816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:55.917808 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:56.015702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fac16-8178-4d9e-abd8-633cab2194c9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:56.022764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:56.023337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:56.023625 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:56.024118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:56.026223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:56.040982 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:56.042605 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:56.054002 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1366/2764] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:42:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:42:57.128703 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:57.129185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:57.129296 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:57.176643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] MFA Rules not processed for user `b1f2a519e5cc42a993a6007174343edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:42:57.177881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:57.241786 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1399/2765] 10.222.0.25 () {58 vars in 967 bytes} [Tue Jul 12 14:42:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:42:57.282435 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:42:57.283841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:42:57.372892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27146c-ec8a-49ad-a1ae-76ed7cd8be51 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:42:57.379617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f47a2120e91452ebcaa6e042caed9ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:42:57.380250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:42:57.380533 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:42:57.380810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:42:57.388764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:42:57.390349 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:42:57.399395 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1367/2766] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Jul 12 14:42:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 14:43:04.092414 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.093969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.104752 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:04.161079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb8f64-bfea-4585-b3c3-09a91f6391bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.165672 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.166548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.166836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.167114 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] PATH_INFO: `/v3/users/2a821209feb04c0a9f67aa6dfe7fb1e9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.168320 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a821209feb04c0a9f67aa6dfe7fb1e9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.170308 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.216588 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a821209feb04c0a9f67aa6dfe7fb1e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.224230 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a821209feb04c0a9f67aa6dfe7fb1e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.236927 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a821209feb04c0a9f67aa6dfe7fb1e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.237601 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a821209feb04c0a9f67aa6dfe7fb1e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.245523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a821209feb04c0a9f67aa6dfe7fb1e9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.251982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1400/2767] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:43:04 2022] DELETE /identity/v3/users/2a821209feb04c0a9f67aa6dfe7fb1e9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:04.442337 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.454212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb7fb6-99d7-488e-8ec2-74e534bd9962 service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.507290 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:04.514714 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.516083 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.516362 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.516634 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] PATH_INFO: `/v3/projects/51b448bb6d0542e9aeb90961739d9c06` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.517831 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b448bb6d0542e9aeb90961739d9c06)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.519967 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.566084 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b448bb6d0542e9aeb90961739d9c06', 'request_id': 'req-b0161128-c36a-4335-a958-09b7342573fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.574160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b448bb6d0542e9aeb90961739d9c06', 'request_id': 'req-b0161128-c36a-4335-a958-09b7342573fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.580265 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b448bb6d0542e9aeb90961739d9c06', 'request_id': 'req-b0161128-c36a-4335-a958-09b7342573fd'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.591030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] The token cache is being invalidate because project 51b448bb6d0542e9aeb90961739d9c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:43:04.591315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.591932 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.593523 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1368/2768] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:43:04 2022] DELETE /identity/v3/projects/51b448bb6d0542e9aeb90961739d9c06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:04.598129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.599654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.613129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5597a1-e3d0-432c-9e50-4b1940ce305a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:04.668493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:04.669390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:04.669681 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:04.669960 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] PATH_INFO: `/v3/users/06bf933dbd03458ab93fb9f598cc522e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:04.671154 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bf933dbd03458ab93fb9f598cc522e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:04.673110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:04.707293 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:04.723079 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bf933dbd03458ab93fb9f598cc522e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.731518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bf933dbd03458ab93fb9f598cc522e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.744355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bf933dbd03458ab93fb9f598cc522e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.745005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bf933dbd03458ab93fb9f598cc522e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.751811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-deabebac-7d4c-4a1b-979e-96302f2045c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bf933dbd03458ab93fb9f598cc522e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:04.757723 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1401/2769] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:43:04 2022] DELETE /identity/v3/users/06bf933dbd03458ab93fb9f598cc522e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:04.925239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:04.926767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:04.942176 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0161128-c36a-4335-a958-09b7342573fd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:05.001051 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:05.001861 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:05.002153 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:05.002434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] PATH_INFO: `/v3/projects/04c862f2784c4a999e5c19c6826bdb6c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:05.003620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c862f2784c4a999e5c19c6826bdb6c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:05.005548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:05.050827 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c862f2784c4a999e5c19c6826bdb6c', 'request_id': 'req-1908f80d-88ca-4345-a1a1-2d8ad7847b47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:05.058664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c862f2784c4a999e5c19c6826bdb6c', 'request_id': 'req-1908f80d-88ca-4345-a1a1-2d8ad7847b47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:05.064416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c862f2784c4a999e5c19c6826bdb6c', 'request_id': 'req-1908f80d-88ca-4345-a1a1-2d8ad7847b47'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:05.075574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] The token cache is being invalidate because project 04c862f2784c4a999e5c19c6826bdb6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:43:05.075866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:05.076504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:43:05.078582 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1369/2770] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:43:04 2022] DELETE /identity/v3/projects/04c862f2784c4a999e5c19c6826bdb6c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:43:14.217280 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:14.217840 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:14.217944 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:14.236463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] MFA Rules not processed for user `64206d561ee4456a8f8c670dfd2944a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:43:14.237727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:14.294894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1402/2771] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 14:43:14.616041 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:14.617559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:14.636456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:14.677507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908f80d-88ca-4345-a1a1-2d8ad7847b47 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:14.682667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:14.682667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:14.682667 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:14.683595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:14.685569 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:14.698250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:14.699709 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:14.710697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1370/2772] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:15.279075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:15.280641 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:15.352938 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43f34b-131c-4f6a-9495-c2040e712d12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:15.357906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:15.358596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:15.358882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:15.359161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:15.360929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:15.372773 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:15.374959 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:15.385266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1403/2773] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:24.705352 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=19 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:25.380795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=75,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:43:26.783504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:43:26.785020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:26.863739 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576da3d-0b7d-40ff-bd6d-5d71399f708d service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:43:26.869455 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:43:26.870112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:43:26.870400 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:43:26.870683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:43:26.872486 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:43:26.885274 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:43:26.886828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:43:26.898344 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1371/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:43:36.893754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=34 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:31.197770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.199181 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.212647 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea0955-74d7-46cd-9c81-adf67e3ca421 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.218367 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.219014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.219300 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.219581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.221426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.233502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.235278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.247132 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1404/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:31.258538 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.260016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.332246 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1a8f1-16e6-48a0-9cbb-c3b30670f28f service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.336762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.337638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.337922 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.338196 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] PATH_INFO: `/v3/users/64206d561ee4456a8f8c670dfd2944a5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.339373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=64206d561ee4456a8f8c670dfd2944a5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.341280 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.386574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64206d561ee4456a8f8c670dfd2944a5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.393374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64206d561ee4456a8f8c670dfd2944a5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.393994 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64206d561ee4456a8f8c670dfd2944a5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.406156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64206d561ee4456a8f8c670dfd2944a5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.414360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-992a7a15-73af-4e74-8bb2-b81a9deab1ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64206d561ee4456a8f8c670dfd2944a5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.421003 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1372/2776] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:44:31 2022] DELETE /identity/v3/users/64206d561ee4456a8f8c670dfd2944a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:31.599393 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.610732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3b08d-1816-4a30-bb34-81dc1b322af8 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.666360 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.667145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.667428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.667710 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] PATH_INFO: `/v3/projects/0d418ed7f92744978dd6a9f1f77287c5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.669437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d418ed7f92744978dd6a9f1f77287c5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.671323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.716141 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d418ed7f92744978dd6a9f1f77287c5', 'request_id': 'req-9e4a0196-283d-45f3-b50e-8cddd199f4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.724519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d418ed7f92744978dd6a9f1f77287c5', 'request_id': 'req-9e4a0196-283d-45f3-b50e-8cddd199f4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.730245 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d418ed7f92744978dd6a9f1f77287c5', 'request_id': 'req-9e4a0196-283d-45f3-b50e-8cddd199f4f9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.741847 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] The token cache is being invalidate because project 0d418ed7f92744978dd6a9f1f77287c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:44:31.742129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.742741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9e4a0196-283d-45f3-b50e-8cddd199f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:44:31.744288 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1405/2777] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:44:31 2022] DELETE /identity/v3/projects/0d418ed7f92744978dd6a9f1f77287c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:31.749498 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.749836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.750272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.768740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:31.769966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.831527 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1373/2778] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:31.839311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.839598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.839874 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.858779 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:31.859963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:31.867890 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1406/2779] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:31.871911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.881578 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d79d-a7cb-4a73-91f2-041936cb5134 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.886140 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.886783 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.887068 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.887341 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.888066 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:44:31.888374 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.889998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.901457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1374/2780] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:44:31.906283 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.915558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b88ca-92f7-4f1e-b35b-e70857342db2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.920781 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:31.921394 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:31.921675 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:31.921963 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:31.922701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:31.924742 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:31.959065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1407/2781] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:44:31 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:44:31.963255 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:31.973086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa57702-e6c4-47b4-b606-41a1f9b83aa0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:31.976643 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:32.030947 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.031599 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.031883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.032156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.032842 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.034773 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.053493 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1375/2782] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:44:31 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:44:32.057598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.068008 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5104ead-6b9e-46b4-9062-1957c41bf022 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.072461 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.073109 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.073440 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.073702 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.074905 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.076487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.086084 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1408/2783] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:32.089747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.100298 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e08bf-6edf-4c30-baa3-f55ba27f48f5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.104878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.105558 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.105878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.106156 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.106883 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.108448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90435-cec7-467e-bd7d-2f4157ee8dec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.118216 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1376/2784] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:32.122350 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.132210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80d2d-9864-4a12-af48-a23eb66ada4d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.137443 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.138319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.138604 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.138881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] PATH_INFO: `/v3/projects/e9f66ee55a2446dca70c24d0384e25f3/users/eeedbd6e02214dbf9edaf19526103266/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.153179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f66ee55a2446dca70c24d0384e25f3, user_id=eeedbd6e02214dbf9edaf19526103266, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.155853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.169144 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1409/2785] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:44:32 2022] PUT /identity/v3/projects/e9f66ee55a2446dca70c24d0384e25f3/users/eeedbd6e02214dbf9edaf19526103266/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:44:32.180086 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.180316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.180595 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.207202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] MFA Rules not processed for user `eeedbd6e02214dbf9edaf19526103266`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:32.208393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:32.271133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1377/2786] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:32.275862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.281757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:32.339600 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3006-bb6a-4905-8bfd-2ba77db4a5bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.344830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.345537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.345821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.346099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.346795 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.348749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.374978 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1410/2787] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:44:32 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:44:32.378902 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.389959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c780c-f59f-4a82-92ce-bf4c08a29ac9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.450097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.450767 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.451052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.451332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.452053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.453992 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.474239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1378/2788] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:44:32 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:44:32.478215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.489251 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802923ce-eb66-45a0-af0b-0c8535683265 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.493895 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.494554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.494849 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.495113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.495861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.497997 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.508451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1411/2789] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:32.513052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.519366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:32.523596 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7495b-d13b-4a10-a455-550fc9b65de2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.529157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.530107 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.530396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.530675 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] PATH_INFO: `/v3/projects/11d8ab338ac342078ec2a85ca6b43750/users/0d52a73fb9754d22a83344b524eccd15/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.545297 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d8ab338ac342078ec2a85ca6b43750, user_id=0d52a73fb9754d22a83344b524eccd15, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.547937 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.560892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1379/2790] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:44:32 2022] PUT /identity/v3/projects/11d8ab338ac342078ec2a85ca6b43750/users/0d52a73fb9754d22a83344b524eccd15/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:32.565033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.575724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8ccc-b00d-47c6-9b4e-c9ac69fc11b1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.630270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.630929 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.631215 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.631489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.632200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.633817 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa465-7a80-4b31-9d14-36013bf55ace admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.643708 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1412/2791] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:44:32.647744 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.657907 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020703ef-ba7a-476f-9750-f38817223a00 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.662496 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.663370 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.663653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.663934 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] PATH_INFO: `/v3/projects/11d8ab338ac342078ec2a85ca6b43750/users/0d52a73fb9754d22a83344b524eccd15/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.665913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d8ab338ac342078ec2a85ca6b43750, user_id=0d52a73fb9754d22a83344b524eccd15, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.669012 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.679272 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1380/2792] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:44:32 2022] PUT /identity/v3/projects/11d8ab338ac342078ec2a85ca6b43750/users/0d52a73fb9754d22a83344b524eccd15/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:44:32.689952 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.690445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.690726 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.716751 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] MFA Rules not processed for user `0d52a73fb9754d22a83344b524eccd15`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:44:32.717979 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:32.781477 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:32.785532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1413/2793] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:44:32.796658 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.798460 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:44:32.876929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d5f-80e4-465d-8fe1-dec32ae59509 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.880545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.881184 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.881495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.881772 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:32.891424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:32.892836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:32.902650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1381/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:32.973714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:44:32.983256 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772139b8-3739-4dbd-9e54-80d07f17b9a3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:44:32.987021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:44:32.987654 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:44:32.987937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:44:32.988211 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:44:33.043434 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:44:33.044853 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc1397-eb03-462d-bcee-198a8815cd03 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:44:33.053543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1414/2795] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:44:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:44:42.898315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:44:43.049677 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=28 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:11.383042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:11.446220 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc39db-1ae6-4367-92b7-9d4c5ba2ec26 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:11.452022 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:11.452895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:11.453207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:11.453522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] PATH_INFO: `/v3/users/eeedbd6e02214dbf9edaf19526103266` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:11.454740 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeedbd6e02214dbf9edaf19526103266)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:11.456668 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:11.507183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeedbd6e02214dbf9edaf19526103266'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:11.514826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeedbd6e02214dbf9edaf19526103266'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:11.515454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeedbd6e02214dbf9edaf19526103266'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:11.527741 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeedbd6e02214dbf9edaf19526103266'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:11.535719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeedbd6e02214dbf9edaf19526103266'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:11.541212 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1382/2796] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:11 2022] DELETE /identity/v3/users/eeedbd6e02214dbf9edaf19526103266 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:11.545906 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:11.546413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:11.546537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:11.566305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:11.567530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:11.627507 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1415/2797] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:11.637865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:11.639397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:11.706456 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f80c-4726-4834-88b5-1421304f6cd3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:11.710261 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:11.710909 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:11.711194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:11.711475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:11.720998 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:11.722717 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:11.732823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1383/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:12.015879 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.018104 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.034537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef495c03-24e3-4687-91d9-fa8966a77c28 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.040473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.041476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.042179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.042486 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] PATH_INFO: `/v3/projects/e9f66ee55a2446dca70c24d0384e25f3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.043731 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f66ee55a2446dca70c24d0384e25f3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.045696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.094472 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f66ee55a2446dca70c24d0384e25f3', 'request_id': 'req-a1dbe1ed-91a7-443e-bb23-01857ade9630'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.102830 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f66ee55a2446dca70c24d0384e25f3', 'request_id': 'req-a1dbe1ed-91a7-443e-bb23-01857ade9630'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.109714 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f66ee55a2446dca70c24d0384e25f3', 'request_id': 'req-a1dbe1ed-91a7-443e-bb23-01857ade9630'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.122075 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] The token cache is being invalidate because project e9f66ee55a2446dca70c24d0384e25f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:12.122365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.123020 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.124570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1416/2799] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:12 2022] DELETE /identity/v3/projects/e9f66ee55a2446dca70c24d0384e25f3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:12.132293 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.144604 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d67d8-5528-4ab1-92a6-5b885b136452 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.165311 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:12.205695 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.206557 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.206836 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.207110 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] PATH_INFO: `/v3/users/0d52a73fb9754d22a83344b524eccd15` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.208292 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d52a73fb9754d22a83344b524eccd15)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.210424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.259123 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d52a73fb9754d22a83344b524eccd15'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.265986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d52a73fb9754d22a83344b524eccd15'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.266705 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d52a73fb9754d22a83344b524eccd15'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.279250 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d52a73fb9754d22a83344b524eccd15'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.286294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cb6f9da3-f403-4fa4-a56f-03294ddad972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d52a73fb9754d22a83344b524eccd15'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.292864 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1384/2800] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:12 2022] DELETE /identity/v3/users/0d52a73fb9754d22a83344b524eccd15 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:12.560572 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.562059 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.575861 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe1ed-91a7-443e-bb23-01857ade9630 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.632137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.633009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.633294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.633596 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] PATH_INFO: `/v3/projects/11d8ab338ac342078ec2a85ca6b43750` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.634789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d8ab338ac342078ec2a85ca6b43750)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.637198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.672724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:12.695582 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d8ab338ac342078ec2a85ca6b43750', 'request_id': 'req-b3d36c37-6e21-47e3-8587-37f1790517fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.704179 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d8ab338ac342078ec2a85ca6b43750', 'request_id': 'req-b3d36c37-6e21-47e3-8587-37f1790517fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.710741 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d8ab338ac342078ec2a85ca6b43750', 'request_id': 'req-b3d36c37-6e21-47e3-8587-37f1790517fe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.722873 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] The token cache is being invalidate because project 11d8ab338ac342078ec2a85ca6b43750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:12.723184 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.724161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b3d36c37-6e21-47e3-8587-37f1790517fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:12.725814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1417/2801] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:12 2022] DELETE /identity/v3/projects/11d8ab338ac342078ec2a85ca6b43750 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:12.730718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.731192 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.731314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.754119 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:12.755368 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.818069 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1385/2802] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:12.827491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.827782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.828068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.846713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:12.847953 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:12.856198 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1418/2803] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:12.860860 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.871266 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e516-5729-490c-a568-c787e829f80a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.875997 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.876650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.876930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.877209 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.877983 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:12.878289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.879892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.892386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1386/2804] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:12.897284 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.907975 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1e499-7cb4-4b6a-a591-0ec9d393d174 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.913065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:12.913765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:12.914077 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:12.914354 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:12.915055 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:12.916990 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:12.952902 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1419/2805] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:12 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:12.957553 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:12.968235 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd10664-8928-4916-9903-27a35ebe7b13 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:12.993807 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:13.028576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.029249 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.029565 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.029895 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.030600 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.032512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.052754 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1387/2806] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:12 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:13.056894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.068492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095ce7-4599-49c9-a3b7-061e04719ba0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.073774 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.074437 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.074724 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.075121 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.075882 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.077534 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.088469 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1420/2807] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:13.092548 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.102850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f043a0e-6663-434b-9cc8-1dc667efd488 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.107529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.108205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.108495 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.108774 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.109576 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.111171 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640d35e-fe95-4958-bb54-275c9d877e59 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.122238 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1388/2808] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:13.126495 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.137318 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f67d3-5b34-41a3-af4d-8ee9964525ac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.142476 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.143397 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.143679 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.143956 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] PATH_INFO: `/v3/projects/720d64b9e13f467583d1b2f5bf1722aa/users/8067742e354f41a5a5d2cbf6ab384f41/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.158783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=720d64b9e13f467583d1b2f5bf1722aa, user_id=8067742e354f41a5a5d2cbf6ab384f41, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.161480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.175131 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1421/2809] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:13 2022] PUT /identity/v3/projects/720d64b9e13f467583d1b2f5bf1722aa/users/8067742e354f41a5a5d2cbf6ab384f41/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:13.186009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.186304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.186617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.213389 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] MFA Rules not processed for user `8067742e354f41a5a5d2cbf6ab384f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:13.214671 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:13.279122 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1389/2810] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:13.283701 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.344980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c8919-b099-4aa0-98eb-19aec365f0c9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.349683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.350345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.350631 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.350910 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.351611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.353541 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.380426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1422/2811] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:13 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:13.384386 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.395175 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269af0-8119-4fae-9856-21931af63ef1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.460194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.460870 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.461154 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.461512 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.462208 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.464346 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.483881 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1390/2812] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:13 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:13.491487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.502424 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c839c34-6f94-4d2e-b863-ed03f161955a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.507088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.507749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.508035 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.508395 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.509042 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.510696 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.522054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1423/2813] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:13.526376 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.536771 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d821-44ff-45a2-902f-d43ba52c5ae8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.542135 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.543030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.543316 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.543598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] PATH_INFO: `/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95/users/61d1fe28b56b4b75b51202ada797c2ef/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.560170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd1f6af3fae41758c8a0e4b07016f95, user_id=61d1fe28b56b4b75b51202ada797c2ef, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.562888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.578478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1391/2814] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:13 2022] PUT /identity/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95/users/61d1fe28b56b4b75b51202ada797c2ef/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:13.582607 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.593626 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec807f-5400-4567-a2e2-d3412f9f260e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.613918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:13.651487 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.652159 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.652445 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.652730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.653489 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.655090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d519a7-595c-48d1-827d-08cf4cd515be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.665023 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1424/2815] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:13.669919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.675563 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:13.682336 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe29d3c-e450-487a-960a-c34eaa1b85f7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.687986 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.688914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.689202 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.689800 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] PATH_INFO: `/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95/users/61d1fe28b56b4b75b51202ada797c2ef/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.691821 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd1f6af3fae41758c8a0e4b07016f95, user_id=61d1fe28b56b4b75b51202ada797c2ef, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.694507 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.705360 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1392/2816] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:13 2022] PUT /identity/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95/users/61d1fe28b56b4b75b51202ada797c2ef/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:13.716032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.716321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.716598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.743843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] MFA Rules not processed for user `61d1fe28b56b4b75b51202ada797c2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:13.745146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:13.819056 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1425/2817] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:13.830121 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.831850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:13.889790 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6dfc5-f3ab-485d-92d2-6737cd770bd8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:13.893382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:13.894053 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:13.894321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:13.894598 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:13.904225 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:13.905696 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:13.915660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1393/2818] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:13.987690 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:13.997478 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf30dc-3dcf-48ce-b564-618ab79786ae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:14.001118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:14.001794 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:14.002074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:14.002348 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:14.059585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:14.061199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eea2d-cc53-4726-9f02-ccb1e90baa06 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:14.070579 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1426/2819] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:22.868205 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:22.932638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260a7b6-9736-4328-ab6e-ea65036f4a6a service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:22.937207 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:22.938118 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:22.938397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:22.938677 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] PATH_INFO: `/v3/users/8067742e354f41a5a5d2cbf6ab384f41` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:22.940182 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8067742e354f41a5a5d2cbf6ab384f41)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:22.942157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:22.994497 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8067742e354f41a5a5d2cbf6ab384f41'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.002920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8067742e354f41a5a5d2cbf6ab384f41'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.003720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8067742e354f41a5a5d2cbf6ab384f41'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.018170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8067742e354f41a5a5d2cbf6ab384f41'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.024988 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8067742e354f41a5a5d2cbf6ab384f41'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.031778 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1394/2820] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:22 2022] DELETE /identity/v3/users/8067742e354f41a5a5d2cbf6ab384f41 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:23.036869 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:23.037428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:23.037428 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:23.055001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:23.056212 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:23.097896 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:23.117228 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1427/2821] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:23.127632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:23.129436 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:23.143780 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:23.212974 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9d61-96df-452a-9bb0-caba1a74a0d1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:23.216840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:23.217529 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:23.217810 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:23.218087 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:23.227954 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:23.230063 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:23.240332 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1395/2822] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:23.525937 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:23.527491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:23.540825 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8423c9c-bd6e-458e-bfa4-84369ea9e344 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:23.547553 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:23.548024 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:23.548316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:23.548648 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] PATH_INFO: `/v3/projects/720d64b9e13f467583d1b2f5bf1722aa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:23.550082 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720d64b9e13f467583d1b2f5bf1722aa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:23.551877 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:23.603581 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720d64b9e13f467583d1b2f5bf1722aa', 'request_id': 'req-8c8a4bdd-f416-44b7-aa33-376212db9c38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.613659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720d64b9e13f467583d1b2f5bf1722aa', 'request_id': 'req-8c8a4bdd-f416-44b7-aa33-376212db9c38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.619349 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720d64b9e13f467583d1b2f5bf1722aa', 'request_id': 'req-8c8a4bdd-f416-44b7-aa33-376212db9c38'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.630491 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] The token cache is being invalidate because project 720d64b9e13f467583d1b2f5bf1722aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:23.630777 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.631426 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.632977 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1428/2823] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:23 2022] DELETE /identity/v3/projects/720d64b9e13f467583d1b2f5bf1722aa => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:23.636953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:23.646796 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5dca-d5a2-4899-a86a-c96a3b01c0ce service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:23.704673 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:23.705573 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:23.705858 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:23.706134 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] PATH_INFO: `/v3/users/61d1fe28b56b4b75b51202ada797c2ef` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:23.707317 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d1fe28b56b4b75b51202ada797c2ef)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:23.709302 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:23.755843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d1fe28b56b4b75b51202ada797c2ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.763915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d1fe28b56b4b75b51202ada797c2ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.764545 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d1fe28b56b4b75b51202ada797c2ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.777221 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d1fe28b56b4b75b51202ada797c2ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.784042 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ac4bee71-d8ac-4346-b998-7d7f2ea86013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d1fe28b56b4b75b51202ada797c2ef'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:23.790172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1396/2824] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:23 2022] DELETE /identity/v3/users/61d1fe28b56b4b75b51202ada797c2ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:24.075034 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.076543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:24.102113 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a4bdd-f416-44b7-aa33-376212db9c38 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.160205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.161080 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.161388 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.161670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] PATH_INFO: `/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.162859 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd1f6af3fae41758c8a0e4b07016f95)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.164765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.210727 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd1f6af3fae41758c8a0e4b07016f95', 'request_id': 'req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:24.218837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd1f6af3fae41758c8a0e4b07016f95', 'request_id': 'req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:24.224403 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd1f6af3fae41758c8a0e4b07016f95', 'request_id': 'req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:24.232064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:24.237278 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] The token cache is being invalidate because project 1cd1f6af3fae41758c8a0e4b07016f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:24.237605 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:24.238519 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67e6e0b2-f065-47f5-99d8-00a5cf8f9f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:24.240554 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1429/2825] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:24 2022] DELETE /identity/v3/projects/1cd1f6af3fae41758c8a0e4b07016f95 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:24.245738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.246240 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.246350 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.264330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:24.265566 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:24.295848 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:24.327409 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1397/2826] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:24.335246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.335559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.335837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.356123 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:24.357366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:24.365569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1430/2827] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:24.369829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.379841 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc4a37-a725-4ee5-ac34-9c27a535c2df None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.385108 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.385830 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.386165 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.386587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.387364 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:24.387674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.389296 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.402242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1398/2828] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:24.408517 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.419117 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fd257-89f5-45fc-aac1-dff06579063a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.424003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.424671 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.424949 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.425220 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.426053 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.428026 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.457894 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1431/2829] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:24.462148 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.472666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab2d47-9001-4b3d-9174-ab237db93871 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.528826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.529559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.530180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.530559 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.531268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.533218 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.553016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1399/2830] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:24.557001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.567665 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3262a6b-980f-49bc-8073-9b6574c1a6dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.573054 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.573763 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.574047 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.574327 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.575072 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.576670 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.587619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1432/2831] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:24.592868 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.603587 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4599-474b-49c2-b4e9-a389f58a4f9b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.608491 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.609157 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.609473 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.616321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.616321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.616321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf079e35-3eed-4106-94df-b706e8eadb71 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.622718 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1400/2832] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:45:24.626918 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.637852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d545b2d-b15b-4704-a855-93afaed6d4c0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.642612 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.643505 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.643798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.644088 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] PATH_INFO: `/v3/projects/2aa1cc69594042c0ad24020e1ddceba9/users/64af6287fe6d4162910ec8d2ef76168c/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.659750 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa1cc69594042c0ad24020e1ddceba9, user_id=64af6287fe6d4162910ec8d2ef76168c, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.662470 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.676167 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1433/2833] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:24 2022] PUT /identity/v3/projects/2aa1cc69594042c0ad24020e1ddceba9/users/64af6287fe6d4162910ec8d2ef76168c/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:24.686524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.686843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.687126 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.715425 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] MFA Rules not processed for user `64af6287fe6d4162910ec8d2ef76168c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:24.716654 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:24.783016 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1401/2834] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:24.788093 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.862643 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfad2d1-cd42-4898-b6a8-addd8da46e02 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.867196 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.867854 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.868137 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.868413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.869637 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.871568 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.899822 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1434/2835] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:24.904112 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:24.915454 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bdb2f-c1ed-4b0f-b0f7-3948fc79ebe7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:24.942017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:24.974464 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:24.975160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:24.975441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:24.975719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:24.976430 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:24.978384 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:24.999015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1402/2836] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:24 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:25.003102 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.014448 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67580e1e-010b-403a-8b1f-7e5bc1a4a734 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.019207 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.019908 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.020190 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.020468 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.021194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.022828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.033725 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1435/2837] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:25.038147 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.048520 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22a5c-21a2-4b3e-81cb-ff7b8da83bfe admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.053181 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.054124 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.054451 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.054732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] PATH_INFO: `/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4/users/a9d38dafda264883a17c7603855a7d34/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.070466 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e7009bbb848e78d264c63b8b8b8c4, user_id=a9d38dafda264883a17c7603855a7d34, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.073732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.087026 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1403/2838] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:25 2022] PUT /identity/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4/users/a9d38dafda264883a17c7603855a7d34/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:25.091930 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.107644 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5bdc7-2e8d-4a8d-9f34-fb163b4d6a56 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.164118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.164713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.165001 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.165281 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.166537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.168149 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb67de-0244-40f8-a56b-8803d817668f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.178118 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1436/2839] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:25.182930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.193579 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a68cb-4f01-4f80-820e-78ee65d04603 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.198139 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.199015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.199295 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.199568 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] PATH_INFO: `/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4/users/a9d38dafda264883a17c7603855a7d34/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.201575 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e7009bbb848e78d264c63b8b8b8c4, user_id=a9d38dafda264883a17c7603855a7d34, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.204172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.214840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1404/2840] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:25 2022] PUT /identity/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4/users/a9d38dafda264883a17c7603855a7d34/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:25.279523 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.280033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.280357 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.288223 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:25.311192 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] MFA Rules not processed for user `a9d38dafda264883a17c7603855a7d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:25.312416 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:25.513999 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1437/2841] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:25.526159 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:25.527850 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:25.603806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5a4a-db5e-4e9c-b7f0-e75b3166321b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:25.609919 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:25.610960 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:25.611431 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:25.611920 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:25.628366 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:25.631039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c68002-d47b-4644-a38c-8edaf7c35912 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:25.649522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1405/2842] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:26.056789 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:26.117319 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549c8d-1143-441f-88b5-1b1bf241280c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:26.122573 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:26.123449 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:26.123730 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:26.124003 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] PATH_INFO: `/v3/users/64af6287fe6d4162910ec8d2ef76168c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:26.125209 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=64af6287fe6d4162910ec8d2ef76168c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:26.127213 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:26.173210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64af6287fe6d4162910ec8d2ef76168c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.179796 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64af6287fe6d4162910ec8d2ef76168c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.191852 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64af6287fe6d4162910ec8d2ef76168c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.192422 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64af6287fe6d4162910ec8d2ef76168c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.199570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64af6287fe6d4162910ec8d2ef76168c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.205306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1438/2843] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:26 2022] DELETE /identity/v3/users/64af6287fe6d4162910ec8d2ef76168c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:26.209854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:26.210142 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:26.210421 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:26.227334 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:26.229031 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:26.286621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1406/2844] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:26.296862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:26.298262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:26.309537 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:26.357446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853219-c40d-41bd-a204-67e5938f65fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:26.361074 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:26.361678 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:26.361962 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:26.362237 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:26.371199 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:26.372611 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:26.381851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1439/2845] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:26.641617 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:26.643061 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:26.658278 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29143-c673-40c9-8ccf-a368bb3a45cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:26.663702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:26.664712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:26.665029 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:26.665263 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] PATH_INFO: `/v3/projects/2aa1cc69594042c0ad24020e1ddceba9` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:26.667074 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa1cc69594042c0ad24020e1ddceba9)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:26.668953 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:26.691457 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:26.716062 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa1cc69594042c0ad24020e1ddceba9', 'request_id': 'req-61eb6b5a-65aa-4ecf-9e57-41408ee75774'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.723377 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa1cc69594042c0ad24020e1ddceba9', 'request_id': 'req-61eb6b5a-65aa-4ecf-9e57-41408ee75774'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.729282 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa1cc69594042c0ad24020e1ddceba9', 'request_id': 'req-61eb6b5a-65aa-4ecf-9e57-41408ee75774'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.744034 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] The token cache is being invalidate because project 2aa1cc69594042c0ad24020e1ddceba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:26.744888 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.745593 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.747120 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1407/2846] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:26 2022] DELETE /identity/v3/projects/2aa1cc69594042c0ad24020e1ddceba9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:26.751069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:26.761030 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7aba0-ca61-41f9-8ff6-e07921b91bc9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:26.814616 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:26.815482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:26.815762 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:26.816033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] PATH_INFO: `/v3/users/a9d38dafda264883a17c7603855a7d34` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:26.817200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d38dafda264883a17c7603855a7d34)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:26.819331 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:26.864323 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d38dafda264883a17c7603855a7d34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.871860 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d38dafda264883a17c7603855a7d34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.883255 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d38dafda264883a17c7603855a7d34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.883878 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d38dafda264883a17c7603855a7d34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.890619 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-74507709-cbdc-4b57-87a8-331bff051454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d38dafda264883a17c7603855a7d34'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:26.896286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1440/2847] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:26 2022] DELETE /identity/v3/users/a9d38dafda264883a17c7603855a7d34 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:27.161294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.162753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:27.176343 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb6b5a-65aa-4ecf-9e57-41408ee75774 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.232844 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.233745 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.234030 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.234307 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] PATH_INFO: `/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.235475 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7e7009bbb848e78d264c63b8b8b8c4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.237393 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.283170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7e7009bbb848e78d264c63b8b8b8c4', 'request_id': 'req-177ab5d4-af40-4250-a9d3-ca05ac9d7024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:27.290172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7e7009bbb848e78d264c63b8b8b8c4', 'request_id': 'req-177ab5d4-af40-4250-a9d3-ca05ac9d7024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:27.295704 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7e7009bbb848e78d264c63b8b8b8c4', 'request_id': 'req-177ab5d4-af40-4250-a9d3-ca05ac9d7024'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:27.306378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] The token cache is being invalidate because project 6e7e7009bbb848e78d264c63b8b8b8c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:27.307164 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:27.307779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-177ab5d4-af40-4250-a9d3-ca05ac9d7024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:27.309285 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1408/2848] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:27 2022] DELETE /identity/v3/projects/6e7e7009bbb848e78d264c63b8b8b8c4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:27.315567 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.316060 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.316180 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.334111 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:27.335305 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:27.361765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:27.396032 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1441/2849] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:27.406260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.406260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.406260 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.424160 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:27.425397 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:27.433399 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1409/2850] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:27.437570 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.447875 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017587f1-5185-48ea-aa9f-a4f17d8f7358 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.452667 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.453311 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.453625 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.453903 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.454622 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:27.454928 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.456506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.468316 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1442/2851] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:27.472416 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.482281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2e8ba-7d33-4d7c-9153-207c3d6cbc52 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.486966 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.487606 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.487890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.488168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.488866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.490822 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.516719 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1410/2852] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:27.520548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.530518 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe99c5-e1de-4791-8b0f-afa38d950430 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.587848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.589143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.589473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.589757 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.590451 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.592362 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.611738 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1443/2853] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:27.615788 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.626172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e44b49-0b2a-4d01-b17b-262c99562f2e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.630761 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.631407 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.631684 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.631957 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.632664 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.634396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.637021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:27.646823 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1411/2854] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:27.651492 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.661782 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f2bc8-3c51-4db4-a98f-f3010ada621e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.666372 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.667009 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.667307 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.667563 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.668286 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.670064 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5787b07-9ba6-4694-a34a-bc2d0e87043e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.680014 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1444/2855] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:27.687949 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.698849 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93057ba1-1ada-410c-ba2e-646a76e52ccb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.703647 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.704547 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.704832 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.705111 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] PATH_INFO: `/v3/projects/d9c58237bf14462e9e0adf82c7e073af/users/3d7b77dd79f34750ae8d06cc97b41b7f/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.720210 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c58237bf14462e9e0adf82c7e073af, user_id=3d7b77dd79f34750ae8d06cc97b41b7f, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.722866 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.736294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1412/2856] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:27 2022] PUT /identity/v3/projects/d9c58237bf14462e9e0adf82c7e073af/users/3d7b77dd79f34750ae8d06cc97b41b7f/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:45:27.746516 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.746811 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.747094 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.774446 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] MFA Rules not processed for user `3d7b77dd79f34750ae8d06cc97b41b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:27.776218 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:27.839006 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1445/2857] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:27.843406 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.904574 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c55dc-b1b9-4241-a424-6b2b72191934 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.909129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:27.909814 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:27.910098 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:27.910373 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:27.911052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:27.912930 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:27.937913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1413/2858] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:27.942383 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:27.953145 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba25b7-43f4-4919-992e-c49830665525 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:27.976655 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:28.008150 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.008804 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.009083 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.009379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.010091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.011971 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.031219 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1446/2859] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:27 2022] POST /identity/v3/users => generated 300 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:28.035195 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.045528 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257819e3-b401-4313-a81f-8ce6aa68d061 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.050223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.050877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.051166 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.051448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.052170 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.053961 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.065268 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1414/2860] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:28.069493 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.079530 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bc27f-be9c-4425-a649-db375f1a3461 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.084161 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.085036 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.085321 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.085633 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] PATH_INFO: `/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8/users/4817d56600ce4fbcb9cc4411d5bb8708/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.100494 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=707b4ba6063a4dfcbad14a5d4b65acf8, user_id=4817d56600ce4fbcb9cc4411d5bb8708, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.103138 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.119151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1447/2861] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:28 2022] PUT /identity/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8/users/4817d56600ce4fbcb9cc4411d5bb8708/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:28.122762 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.132911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d3c45-494e-4511-b6bc-91434dc59a17 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.188973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.189915 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.190197 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.190472 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.191186 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.192764 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc225f-d8c0-489f-bdb4-0a0410038781 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.202972 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1415/2862] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:28.207019 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.217465 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a101-c191-49da-b3c3-c38479add52c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.222110 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.222980 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.223266 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.223564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] PATH_INFO: `/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8/users/4817d56600ce4fbcb9cc4411d5bb8708/roles/640b98e8829b4de0bdceee05e74bf473` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.225550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707b4ba6063a4dfcbad14a5d4b65acf8, user_id=4817d56600ce4fbcb9cc4411d5bb8708, role_id=640b98e8829b4de0bdceee05e74bf473)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.228157 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.238784 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1448/2863] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:28 2022] PUT /identity/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8/users/4817d56600ce4fbcb9cc4411d5bb8708/roles/640b98e8829b4de0bdceee05e74bf473 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:28.250715 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.251010 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.251294 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.278133 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] MFA Rules not processed for user `4817d56600ce4fbcb9cc4411d5bb8708`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:28.279564 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:28.312094 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:28.349615 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1416/2864] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:28.360216 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.362272 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:28.420254 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a302b5-c826-4cdd-aa23-ca8e66604999 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.423916 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.424560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.424862 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.425136 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.434651 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.436065 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a8d12-b873-4445-9985-855a811a5220 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.445683 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1449/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:28.572843 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.636028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa97df-b272-4a01-aa06-754478051a7a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.640448 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.641313 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.641630 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.641910 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] PATH_INFO: `/v3/users/3d7b77dd79f34750ae8d06cc97b41b7f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.643097 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7b77dd79f34750ae8d06cc97b41b7f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.645015 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.695116 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7b77dd79f34750ae8d06cc97b41b7f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.702508 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7b77dd79f34750ae8d06cc97b41b7f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.703233 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7b77dd79f34750ae8d06cc97b41b7f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.715526 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7b77dd79f34750ae8d06cc97b41b7f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.722283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7b77dd79f34750ae8d06cc97b41b7f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:28.728150 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1417/2866] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:28 2022] DELETE /identity/v3/users/3d7b77dd79f34750ae8d06cc97b41b7f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:28.733226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.733545 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.733827 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.750640 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:28.751833 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:28.809155 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1450/2867] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:28.819017 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:28.820403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:28.873632 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:28.877878 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a95-0078-410d-ba5d-b2ceaafe475d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:28.882577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:28.883315 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:28.883670 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:28.884115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:28.893577 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:28.894928 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:28.904976 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1418/2868] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:29.171497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.173021 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.192413 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346357-47f1-4bab-a508-abcefd70a349 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.197969 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.198828 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.199129 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.199427 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] PATH_INFO: `/v3/projects/d9c58237bf14462e9e0adf82c7e073af` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.200560 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c58237bf14462e9e0adf82c7e073af)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.202575 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.218946 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:29.249621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c58237bf14462e9e0adf82c7e073af', 'request_id': 'req-fd99efa0-5c09-417c-9eca-260f09187f30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.256965 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c58237bf14462e9e0adf82c7e073af', 'request_id': 'req-fd99efa0-5c09-417c-9eca-260f09187f30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.262715 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c58237bf14462e9e0adf82c7e073af', 'request_id': 'req-fd99efa0-5c09-417c-9eca-260f09187f30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.273400 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] The token cache is being invalidate because project d9c58237bf14462e9e0adf82c7e073af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:29.273687 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.274303 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.275857 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1451/2869] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:29 2022] DELETE /identity/v3/projects/d9c58237bf14462e9e0adf82c7e073af => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:29.280021 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.290168 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccc780-d164-4916-96db-84be9d686e18 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.346067 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.346935 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.347222 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.347504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] PATH_INFO: `/v3/users/4817d56600ce4fbcb9cc4411d5bb8708` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.348710 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4817d56600ce4fbcb9cc4411d5bb8708)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.350702 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.396452 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4817d56600ce4fbcb9cc4411d5bb8708'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.405382 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4817d56600ce4fbcb9cc4411d5bb8708'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.405973 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4817d56600ce4fbcb9cc4411d5bb8708'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.419518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4817d56600ce4fbcb9cc4411d5bb8708'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.426281 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-40af2a64-16f9-4409-aab1-fb733ffa3bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4817d56600ce4fbcb9cc4411d5bb8708'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.432438 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1419/2870] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:29 2022] DELETE /identity/v3/users/4817d56600ce4fbcb9cc4411d5bb8708 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:29.701069 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.702558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.716514 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99efa0-5c09-417c-9eca-260f09187f30 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.772968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.773876 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.774162 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.774506 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] PATH_INFO: `/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.775765 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=707b4ba6063a4dfcbad14a5d4b65acf8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.777712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:29.822839 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707b4ba6063a4dfcbad14a5d4b65acf8', 'request_id': 'req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.830091 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707b4ba6063a4dfcbad14a5d4b65acf8', 'request_id': 'req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.835669 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707b4ba6063a4dfcbad14a5d4b65acf8', 'request_id': 'req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.846379 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] The token cache is being invalidate because project 707b4ba6063a4dfcbad14a5d4b65acf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:29.846668 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.847294 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-21f84e2b-6cfa-48e3-9ac0-89114d7c48bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:29.848806 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1452/2871] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:29 2022] DELETE /identity/v3/projects/707b4ba6063a4dfcbad14a5d4b65acf8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:29.854355 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.854959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.854959 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.873931 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:29.875279 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.934759 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1420/2872] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:29.942805 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.943099 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.943432 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.961420 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:29.962621 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:29.970482 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1453/2873] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:29.974732 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:29.985020 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ef6c9-5471-46c9-9fac-f0169b31494e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:29.989789 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:29.990442 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:29.990727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:29.991009 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:29.991753 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:29.992048 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:29.993700 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.001000 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:30.008323 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1421/2874] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 14:45:30.012630 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.021912 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeacab-e132-4635-8af5-0a7093483490 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.026548 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.027194 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.027480 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.027759 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.028455 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.030390 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.056090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1454/2875] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:30 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:30.060201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.070635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648e7be-f3b5-4b08-8cf5-fe87634a30db admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.126434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.127103 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.127387 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.127666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.128359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.130299 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.150840 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1422/2876] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:30 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:30.154558 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.165193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370848e-89d5-4242-9bb4-8c6c88e5d0dc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.170345 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.170996 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.171276 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.171552 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.172277 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.173992 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.175376 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:30.190430 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1455/2877] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:30.191891 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.202712 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789d721-b9ef-4df3-9e25-8f46def409e2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.207242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.207892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.208183 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.208468 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.209194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.210826 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79aa9d-086c-4cda-a37b-3c38cf715ecf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.221459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1423/2878] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:30.225423 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.235226 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b6cf8-d0c2-4db7-a329-31f4a96196c7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.239848 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.240713 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.240995 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.241267 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] PATH_INFO: `/v3/projects/2b1991bbf8394e9b8b2815dc3a27397a/users/82467117d3d54e3e8f6cb5e26072868f/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.256749 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1991bbf8394e9b8b2815dc3a27397a, user_id=82467117d3d54e3e8f6cb5e26072868f, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.259585 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.272322 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1456/2879] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:30 2022] PUT /identity/v3/projects/2b1991bbf8394e9b8b2815dc3a27397a/users/82467117d3d54e3e8f6cb5e26072868f/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:30.284683 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.284980 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.285283 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.311471 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] MFA Rules not processed for user `82467117d3d54e3e8f6cb5e26072868f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:30.312674 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:30.378727 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1424/2880] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:30.391007 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:30.392695 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:30.449595 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1d4bb-e950-4598-93e7-d128b60f8a6d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:30.453359 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:30.453987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:30.454270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:30.454543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:30.463769 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:30.465172 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31154145-33ce-4df7-9c9e-fbf59b499ce1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:30.473936 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1457/2881] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:40.375650 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:40.469945 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:47.911892 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:47.973462 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760752a1-ce2a-45f2-b6de-b25388555f78 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:47.978052 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:47.978905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:47.979201 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:47.979635 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] PATH_INFO: `/v3/users/82467117d3d54e3e8f6cb5e26072868f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:47.980890 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82467117d3d54e3e8f6cb5e26072868f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:47.982806 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.027803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82467117d3d54e3e8f6cb5e26072868f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.035793 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82467117d3d54e3e8f6cb5e26072868f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.036289 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82467117d3d54e3e8f6cb5e26072868f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.055467 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82467117d3d54e3e8f6cb5e26072868f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.062685 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82467117d3d54e3e8f6cb5e26072868f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.068766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1425/2882] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:45:47 2022] DELETE /identity/v3/users/82467117d3d54e3e8f6cb5e26072868f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:48.074068 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.074559 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.074783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.091365 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:48.092546 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.148268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1458/2883] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:48.157816 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.159194 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.215608 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9813b155-d68e-41d1-88a1-b265e54c46d2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.219321 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.219948 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.220227 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.220502 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.229618 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.231023 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1031d8b-2983-4ece-9d45-f855127aab78 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.240752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1426/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:48.509463 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.511005 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.523569 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48bb29-4dc8-4d62-b571-25f0abb4071f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.528246 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.529193 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.529510 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.529785 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] PATH_INFO: `/v3/projects/2b1991bbf8394e9b8b2815dc3a27397a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.531057 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1991bbf8394e9b8b2815dc3a27397a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.532907 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.581431 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1991bbf8394e9b8b2815dc3a27397a', 'request_id': 'req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.588304 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1991bbf8394e9b8b2815dc3a27397a', 'request_id': 'req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.593816 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1991bbf8394e9b8b2815dc3a27397a', 'request_id': 'req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.604987 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] The token cache is being invalidate because project 2b1991bbf8394e9b8b2815dc3a27397a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:45:48.605270 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.605920 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fea66e02-8afe-43d9-ba51-3f7fe175e1ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:45:48.607435 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1459/2885] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:45:48 2022] DELETE /identity/v3/projects/2b1991bbf8394e9b8b2815dc3a27397a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:48.612330 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.612697 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.613075 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.631621 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:48.632828 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.649441 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:48.694679 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1427/2886] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:48.704798 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.705090 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.705407 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.723120 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:48.724317 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:48.732571 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1460/2887] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:48.736660 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.746857 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5c36d-abd8-4e00-a83f-a1174dfbb526 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.751583 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.752223 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.752504 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.752779 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.753524 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 14:45:48.753829 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.755405 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.766865 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1428/2888] 10.222.0.25 () {58 vars in 1152 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 14:45:48.770801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.779837 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f41f9-fb69-421c-af65-499de40de7cb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.784818 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.785508 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.785821 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.786103 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.786783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.788768 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.814973 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1461/2889] 10.222.0.25 () {60 vars in 1152 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 14:45:48.819033 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.829039 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675e15c-17de-41f9-8953-58f1ec75ab50 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.885648 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.886239 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.886522 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.886799 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.887482 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.889543 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.908979 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1429/2890] 10.222.0.25 () {60 vars in 1143 bytes} [Tue Jul 12 14:45:48 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 14:45:48.913177 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.923923 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac83ad-f6ed-4118-87c7-3e9f9cc95e92 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.928515 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.929168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.929502 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.929787 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.930547 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.932151 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.942549 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1462/2891] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 14:45:48.946396 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.956180 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29bd3d-ae58-4e3f-ae1e-f36a6df8077f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.960707 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.961359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.961653 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.961929 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:48.963158 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:48.964713 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6662a-6b28-4b30-881d-81a055dd5896 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:48.974417 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1430/2892] 10.222.0.25 () {58 vars in 1121 bytes} [Tue Jul 12 14:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 14:45:48.978836 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:48.988529 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49feeca-b89d-4878-9f9b-9443dfec0266 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:48.993097 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:48.994540 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:48.994826 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:48.995107 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] PATH_INFO: `/v3/projects/f5422e72a4e64b30af144dd6721a1ebc/users/4a59322ececb422f955e4f583c3fbdbe/roles/37d514b6250f41f1a7a945e02510a6df` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:49.009954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5422e72a4e64b30af144dd6721a1ebc, user_id=4a59322ececb422f955e4f583c3fbdbe, role_id=37d514b6250f41f1a7a945e02510a6df)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:49.012564 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:49.024982 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1463/2893] 10.222.0.25 () {60 vars in 1482 bytes} [Tue Jul 12 14:45:48 2022] PUT /identity/v3/projects/f5422e72a4e64b30af144dd6721a1ebc/users/4a59322ececb422f955e4f583c3fbdbe/roles/37d514b6250f41f1a7a945e02510a6df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:45:49.030130 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd367-b680-4b6b-8af4-41f7a475369c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:49.030424 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd367-b680-4b6b-8af4-41f7a475369c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:49.030706 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd367-b680-4b6b-8af4-41f7a475369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:49.048237 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-00ddd367-b680-4b6b-8af4-41f7a475369c None None] MFA Rules not processed for user `a1d6aa7bda7d44f4ab9f044e2d9e08a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:49.049540 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-00ddd367-b680-4b6b-8af4-41f7a475369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:49.108637 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1431/2894] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:49.118732 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:49.120417 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:49.142543 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:49.177745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf534f0-9ebf-4b5c-bf2c-4a34d5eda2dd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:49.181914 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:49.182557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:49.182843 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:49.183122 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:49.192205 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:49.193673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:49.202954 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1464/2895] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:49.783752 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d09c-27ab-4128-8c90-b510e83cc8ed None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:49.784129 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d09c-27ab-4128-8c90-b510e83cc8ed None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:49.784518 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d09c-27ab-4128-8c90-b510e83cc8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:49.810952 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0672d09c-27ab-4128-8c90-b510e83cc8ed None None] MFA Rules not processed for user `4a59322ececb422f955e4f583c3fbdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:49.812172 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0672d09c-27ab-4128-8c90-b510e83cc8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:49.829911 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:45:49.880532 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1432/2896] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:49.892598 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:45:49.894168 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:49.952248 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095af5-4de2-4871-a240-7514845fdfac service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:45:49.955851 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b29bba6ee54cab9e3f6fdcc841ec77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:45:49.956497 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:49.956783 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:49.957067 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:49.966022 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:45:49.969143 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:45:49.978760 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1465/2897] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:45:51.331763 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:45:51.332286 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:45:51.332398 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:45:51.349620 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] MFA Rules not processed for user `4a59322ececb422f955e4f583c3fbdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:45:51.350803 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:45:51.357743 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1433/2898] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:45:59.974325 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=43 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:00.310399 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:00.311823 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:00.387306 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f8189-dad2-42b2-81d1-b1c07e695fbc service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:00.392770 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fb1196840240b4aa05a2f83d4c7ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:00.393473 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:00.393753 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:00.394025 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:00.395968 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:00.409200 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:00.410845 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6540abc-27dd-477d-9cb3-7a1da6ea60fd service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:00.422532 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1466/2899] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 14:46:01.344427 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=22 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:10.418577 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=34 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:30.838784 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:30.840253 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:30.914877 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33671ea6-c9c2-4ea4-bd36-8f82ae2953f3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:30.920738 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:30.921434 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:30.921720 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:30.922426 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:30.929854 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:30.938242 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:30.941403 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:30.953676 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1434/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:40.948642 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=32 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:46:49.911557 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:49.912031 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:49.912130 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:49.929550 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] MFA Rules not processed for user `4a59322ececb422f955e4f583c3fbdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 14:46:49.930772 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:49.937659 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1467/2901] 10.222.0.25 () {58 vars in 957 bytes} [Tue Jul 12 14:46:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 14:46:50.167273 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:46:50.178040 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c976a54-c040-4bae-aace-12073fc57334 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:46:50.182115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b899731a1ef4d9b961492dd78632ac5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:46:50.182747 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:46:50.183028 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:46:50.183304 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:46:50.185044 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:46:50.195459 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:46:50.196859 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:46:50.204914 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1435/2902] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:46:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:46:59.924745 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:00.201314 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:44.957210 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:44.968801 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611912-eb4e-4770-ab77-e787fa4562dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:44.973883 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:44.974858 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:44.975033 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:44.975342 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] PATH_INFO: `/v3/users/4a59322ececb422f955e4f583c3fbdbe` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:44.976498 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a59322ececb422f955e4f583c3fbdbe)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:44.979340 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:45.026135 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a59322ececb422f955e4f583c3fbdbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.034224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a59322ececb422f955e4f583c3fbdbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.046224 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a59322ececb422f955e4f583c3fbdbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.046881 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a59322ececb422f955e4f583c3fbdbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.054355 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a59322ececb422f955e4f583c3fbdbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.060173 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1468/2903] 10.222.0.25 () {58 vars in 1223 bytes} [Tue Jul 12 14:47:44 2022] DELETE /identity/v3/users/4a59322ececb422f955e4f583c3fbdbe => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 14:47:45.234024 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:47:45.247414 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3eba4-4d6e-4646-9837-13d3f255e2cc service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:47:45.304478 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1d6aa7bda7d44f4ab9f044e2d9e08a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dcdffb6815414f8835106f58a82105', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:47:45.305354 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:47:45.305638 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:47:45.305913 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] PATH_INFO: `/v3/projects/f5422e72a4e64b30af144dd6721a1ebc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:47:45.307073 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5422e72a4e64b30af144dd6721a1ebc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:47:45.308940 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:47:45.360359 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5422e72a4e64b30af144dd6721a1ebc', 'request_id': 'req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.369115 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5422e72a4e64b30af144dd6721a1ebc', 'request_id': 'req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.374666 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5422e72a4e64b30af144dd6721a1ebc', 'request_id': 'req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.385461 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] The token cache is being invalidate because project f5422e72a4e64b30af144dd6721a1ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 14:47:45.385766 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.386378 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-05f9c7c8-87e9-47ec-8c4c-4c42cd835ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 14:47:45.387905 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1436/2904] 10.222.0.25 () {58 vars in 1232 bytes} [Tue Jul 12 14:47:45 2022] DELETE /identity/v3/projects/f5422e72a4e64b30af144dd6721a1ebc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 14:47:55.056673 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:47:55.382490 dv-cinder-836996-39-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 14:51:35.776268 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 14:51:35.777716 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:35.914901 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05635ad2-53ff-438c-97b9-7ebe48e80344 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 14:51:35.973185 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a342fa16a564ef38675936d7f51578b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a95557d6a5450399d77fd04fed233c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 14:51:35.973865 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 14:51:35.974146 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 14:51:35.974419 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 14:51:35.976169 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 14:51:35.987712 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 14:51:35.989262 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397266f8-3655-4cd3-a1df-38ba871ef0a1 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 14:51:35.999366 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1469/2905] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Jul 12 14:51:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 14:51:45.994814 dv-cinder-836996-39-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}